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Abstract 

We use the global stochastic analysis tools introduced by P. A. Meyer and L. Schwartz to write 
down a stochastic generalization of the Hamilton equations on a Poisson manifold that, for exact 
symplectic manifolds, are characterized by a natural critical action principle similar to the one 
encountered in classical mechanics. Several features and examples in relation with the solution 
semimartingales of these equations are presented. 
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1 Introduction 

The generalization of classical mechanics to the context of stochastic dynamics has been an active 
research subject ever since K. Ito introduced the theory of stochastic differential equations in the 1950s 
(see for instance [Ni67l iBSTl lYSTl [ZY821 [ZM841 [T#7l ITZ97al IM [CD061 IBRO071 IBRU07a| . and 
references therein) . The motivations behind some pieces of work related to this field lay in the hope that a 
suitable stochastic generalization of classical mechanics should provide an explanation of the intrinsically 
random effects exhibited by quantum mechanics within the context of the theory of diffusions . In other 
instances the goal is establishing a framework adapted to the handling of mechanical systems subjected 
to random perturbations or whose parameters are not precisely determined and are hence modeled as 
realizations of a random variable. 

Most of the pieces of work in the first category use a class of processes that have a stochastic derivative 
introduced in [Ne67j and that has been subsequently refined over the years. This derivative can be used 
to formulate a real valued action and various associated variational principles whose extremals are the 
processes of interest. 

The approach followed in this paper is closer to the one introduced in [B81| in which the action has 
its image in the space of real valued processes and the variations are taken in the space of processes 
with values in the phase space of the system that we are modeling. This paper can be actually seen as 
a generalization of some of the results in |B81| in the following directions: 

(i) We make extensive use of the global stochastic analysis tools introduced by P. A. Meyer |M81|,[M82] 
and L. Schwartz (Sch82j to handle non-Euclidean phase spaces. This feature not only widens the 
spectrum of systems that can be handled but it is also of paramount importance at the time of 
reducing them with respect to the symmetries that they may eventually have (see |LO07j ): indeed, 
the orbit spaces obtained after reduction are generically non-Euclidean, even if the original phase 
space is. 
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(ii) The stochastic dynamical components of the system are modeled by continuous semimartingales 

and are not limited to Brownian motion. 

(iii) We handle stochastic Hamiltonian systems on Poisson manifolds and not only on symplectic man- 
ifolds. 

(iv) The variational principle that we propose in Theorem 14.141 is not just satisfied by the stochastic 
Hamiltonian equations (as in [B81| ) but fully characterizes them. 

There are various reasons that have lead us to consider these generalized Hamiltonian systems. 
First, even though the laws that govern the dynamics of classical mechanical systems are, in principle, 
completely known, the finite precision of experimental measurements yields impossible the estimation of 
the parameters of a particular given one with total accuracy. Second, the modeling of complex physical 
systems involves most of the time simplifying assumptions or idealizations of parts of the system, some 
of which could be included in the description as a stochastic component; this modeling philosophy has 
been extremely successful in the social sciences |B J76j . Third, even if the model and the parameters 
of the system are known with complete accuracy, the solutions of the associated differential equations 
may be of great complexity and exhibit high sensitivity to the initial conditions hence making the 
probabilistic treatment and description of the solutions appropriate. Finally, we will see (Section 13. 3p 
how stochastic Hamiltonian modeling of microscopic systems can be used to model dissipation and 
macroscopic damping. 

The paper is structured as follows: in Section [2] we introduce the stochastic Hamilton equations 
with phase space a given Poisson manifold and we study some of the fundamental properties of the 
solution semimartingales like, for instance, the preservation of symplectic leaves or the characterization 
of the conserved quantities. This section contains a discussion on two notions on non-linear stability, 
almost sure Lyapunov stability and stability in probability, that reduce in the deterministic setup to 
the standard definition of Lyapunov stability. We formulate criteria that generalize to the Hamiltonian 
stochastic context the standard energy methods to conclude the stability of a Hamiltonian equilibrium 
using existing conservation laws. More specifically, there are two different natural notions of conserved 
quantity in the stochastic context that, via a stochastic Dirichlet criterion fTheorem I2.15|) allow one 
to conclude the different kinds of stability that we have mentioned above. Section [3] contains several 
examples: in the first one we show how the systems studied by Bismut in [B81] fall in the category 
introduced in Section [21 We also see that a damped oscillator can be described as the average motion of 
the solution semimartingale of a natural stochastic Hamiltonian system, and that Brownian motion in 
a manifold is the projection onto the base space of very simple Hamiltonian stochastic semimartingale 
defined on the cotangent bundle of the manifold or of its orthonormal frame bundle, depending on the 
availability or not of a parallelization for the manifold in question. Section |4] is dedicated to showing 
that the stochastic Hamilton equations are characterized by a critical action principle that generalizes 
the one found in the treatment of deterministic systems. In order to make this part more readable, the 
proofs of most of the technical results needed to prove the theorems in this section have been included 
separately at the end of the paper. 

One of the goals of this paper is conveying to the geometric mechanics community the plentitude of 
global tools available to handle mechanical problems that contain a stochastic component and that do 
not seem to have been exploited to the full extent of their potential. In order to facilitate the task of 
understanding the paper to non-probabilists we have included an appendix that provides a self-contained 
presentation of some major facts in stochastic calculus on manifolds needed for a first comprehension 
of our results. Those pages are a very short and superficial presentation of a deep and technical field 
of mathematics so the reader interested in a more complete account is encouraged to check with the 
references quoted in the appendix and especially with the excellent monograph |E89| . 
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Conventions: All the manifolds in this paper are finite dimensional, second-countable, locally compact, 
and HausdorfF (and hence paracompact). 

2 The stochastic Hamilton equations 

In this section we present a natural generalization of the standard Hamilton equations in the stochastic 
context. Even though the arguments gathered in the following paragraphs as motivation for these 
equations are of formal nature, we will see later on that, as it was already the case for the standard 
Hamilton equations, they satisfy a natural variational principle. 

We recall that a symplectic manifold is a pair (M, ut), where M is a manifold and u; G rt^{M) 
is a closed non-degenerate two-form on M, that is, dui = and, for every m e M, the map v G 
TmM I— > u!{m){v, •) G T^M is a linear isomorphism between the tangent space TmM to M at m and 
the cotangent space T^M. Using the nondegeneracy of the symplectic form uj, one can associate each 
function h £ C°°{M) a vector field Xh e X(A/), defined by the equality 



We will say that Xh is the Hamiltonian vector field associated to the Hamiltonian function h. 
The expression (|2.ip is referred to as the Hamilton equations. 

A Poisson manifold is a pair (M, {•,•}), where M is a manifold and {•, •} is a bilinear operation on 
C°°(M) such that (C°°(Af), {•, •}) is a Lie algebra and {•, •} is a derivation (that is, the Leibniz identity 
holds) in each argument. The functions in the center C{M) of the Lie algebra (C°°(M), {•, •}) are called 
Casimir functions. From the natural isomorphism between derivations on C°°{M) and vector fields 
on M it follows that each h G C°°{M) induces a vector field on M via the expression Xh — {-jh}, called 
the Hamiltonian vector field associated to the Hamiltonian function h. Hamilton's equations 
z = Xh{z) can be equivalently written in Poisson bracket form as / = {f,h}, for any / G C°°{M). 
The derivation property of the Poisson bracket implies that for any two functions f,g€ C°°{M), the 
value of the bracket {/, g}{z) at an arbitrary point z E M (and therefore Xf{z) as well), depends on / 
only through df(z) which allows us to define a contravariant antisymmetric two-tensor B G A^(M) by 
B{z){a^, P^) = {/, g}iz), where df{z) = G T*M and dg{z) = G T*M. This tensor is called the 
Poisson tensor of M. The vector bundle map B'^ : T*M — > TM naturally associated to B is defined 



We start by rewriting the solutions of the standard Hamilton equations in a form that we will be able 
to mimic in the stochastic differential equations context. All the necessary prerequisites on stochastic 
calculus on manifolds can be found in a short review in the appendix at the end of the paper. 

Proposition 2.1 Let (M,lo) be a .symplectic manifold and h G C°°{M). The smooth curve 7 : [0,T] — > 
M is an integral curve of the Hamiltonian vector field Xh if and only if for any a G il,{M) and for any 



where uj"^ : T* M TM is the vector bundle isomorphism induced by ui. More generally, if M is a 
Poisson manifold with bracket {•, •} then the same result holds with ^2. 2\) replaced by 



\x^(^ = dh. 



(2.1) 



by B(z)(a„ 13,) = (a„ B«(/3,)). 



i G [0,T] 




(2.2) 




(2.3) 
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Proof. Since in the symplectic case — B^, it suffices to prove (I2.3p . As (|2.3p holds for any t € [0, T], 
we can take derivatives with respect to t on both sides and we obtain the equivalent form 

Hlit)),m - -(dM7W),S«(7W)(«(7W))). (2.4) 
Let / G C°°{M) be such that df{"/{t)) = a{j{t)). Then fM]) can be rewritten as 

(d/(7W),7W) - -(dM7W),i?«(7W)(d/(7W))) = {/, /i}(7(i)), 
which is equivalent to j{t) = Xh{'^{t)), as required. I 

We will now introduce the stochastic Hamilton equations by mimicking in the context of Stratonovich 
integration the integral expressions p.2|) and ()2.3p . In the next definition we will use the following 
notation: let f : M ^ W he a, difFerentiable function that takes values on the vector space W. We define 
the differential df : TM ^ as the map given by df =p2oTf, where Tf : TM TW xW 
is the tangent map of / and p2 : W x W ^ W is the projection onto the second factor. If = M this 
definition coincides with the usual differential. If {ei, . . . ,e„} is a basis of W and / = X]"=i Z*^* then 
d/ = Er=idr®e,. 

Definition 2.2 Let (M, {•,•}) be a Poisson manifold, X : IR+ x Q V a semimartingale that takes 
values on the vector space V with Xq = 0, and h : M ^ V* a smooth function. Let {e^, . . . je*"} he 
a basis of V* and h ~ J2i=i^i^''- Hamilton equations with stochastic component X, and 

Hamiltonian function h are the Stratonovich stochastic differential equation 

6T'' = H{X,T)6X, (2.5) 

defined by the Stratonovich operator H{v, z) : TyV T^M given by 

r 

H{v,z){u):^Y.<e\u)Xh^{z). (2.6) 

The dual Stratonovich operator H*{v,z) : T*M T*V of H(v,z) is given by H*{v,z){az) = 
—dh{z)-B^{z){az)- Hence, the results quoted in Appendix 16 . 41 show that for any measurable random 
variable Fq, there exists a unique semimartingale such that Fq = Fq and a maximal stopping time 
C,^ that solve (1231) . that is, for any a G ri(A/), 

j {a, ST'^) - - 1 {dh{B^{a)){T'^),SX). (2.7) 

We will refer to F'* as the Hamiltonian semimartingale associated to h with initial condition Fq. 

Remark 2.3 The stochastic component X encodes the random behavior exhibited by the stochastic 
Hamiltonian system that we are modeling and the Hamiltonian function h specifies how it embeds in 
its phase space. Unlike the situation encountered in the deterministic setup we allow the Hamiltonian 
function to be vector valued in order to accommodate higher dimensional stochastic dynamics. 

Remark 2.4 The generalization of Hamilton's equations proposed in Definition l2.2l bv using a Stratonovich 
operator is inspired by one of the transfer principles presented in |E90| to provide stochastic versions of 
ordinary differential equations. This procedure can be also used to carry out a similar generalization of 
the equations induced by a Leibniz bracket (see [OP04| ). 
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Remark 2.5 Stratonovich versus Ito integration: at the time of proposing the equations in Def- 
inition 12.21 a choice has been made, namely, we have chosen Stratonovich integration instead of Ito or 
other kinds of stochastic integration. The option that we took is motivated by the fact that by using 
Stratonovich integration, most of the geometric features underlying classical deterministic Hamiltonian 
mechanics are preserved in the stochastic context (see the next section). Additionally, from the math- 
ematical point of view, this choice is the most economical one in the sense that the classical geometric 
ingredients of Hamiltonian mechanics plus a noise semimartingale suffice to construct the equations; 
had we used Ito integration we would have had to provide a Schwartz operator (see Section [631) ^^'^ the 
construction of such an object via a transfer principle like in [E90| involves the choice of a connection. 

The use of Ito integration in the modeling of physical phenomena is sometimes preferred because 
the definition of this integral is not anticipative, that is, it does not assume any knowledge about the 
behavior of the system in future times. Even though we have used Stratonovich integration to write 
down our equations, we also share this feature because the equations in Definition 12.21 can be naturally 
translated to the Ito framework (see Proposition 12. 8p . This is a particular case of a more general 
fact since given any Stratonovich stochastic differential equation there always exists an equivalent Ito 
stochastic differential equation, in the sense that both equations have the same solutions. Note that the 
converse is in general not true. 



2.1 Elementary properties of the stochastic Hamilton's equations 

Proposition 2.6 Let (M, {•, •}) be a Poisson manifold, X : M_|_ x ^ V a semimartingale that takes 
values on the vector space V with Xq ~ and h : M ^ V* a smooth Junction. Let To he a Tq measurable 
random variable and F'' the Hamiltonian semimartingale associated to h with initial condition Tq . Let 
C'' be the corresponding maximal stopping time. Then, for any stopping time r < C'', the Hamiltonian 
semimartingale F'* satisfies 



(2.8) 



where {^j}je{i,...,r} o'^s tf^^ components of h and X with respect to two given dual 

bases {ei,...,er} and {e^,...,e''} of V and V* , respectively. Expression Ii2. 8\) can be rewritten in 
differential notation as 

r 

'5/(r'') = E{/'^^}(r'')^^'- 

i=i 



Proof. It suffices to take a = df in (|2.7p . Indeed, by (|6.5p 

r(d/,5F'') = /(F^!)-/(Fg) 

At the same time 



- r(dMi?«(d/))(n,5X) = -^ r {idh,®e^iBHdfmr'^),5X) =j2 r {{f,h,}{V^y,5X). 

Jo 7=1 "'O 7=1 "'0 



By the second statement in (H^]) this equals Y.]^^ /J"{/, hj}{T^)6 (/ {e\5X)). Given that / {t^ ,6X) 

^0' 



X^ — Xl, the equality follows. 
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Remark 2.7 Notice that if in Definition we take V* = R, h € C°°{M), and X : R+ x ^2 R the 
deterministic process given by (t, ui) i — > t, then the stochastic Hamihon equations (|2.7[) reduce to 

{a,dr'') = / {a,Xh){r^)dt. (2.9) 



A straightforward apphcation of (|2.8p shows that rj'(a;) is necessarily a differentiable curve, for any 
£ ri, and hence the Riemann-Stiehjes integral in the left hand side of (|2.9p reduces, when evaluated 
at a given w £ i7, to a Riemann integral identical to the one in the left hand side of (|2.3p . hence proving 
that (|2.9p reduces to the standard Hamilton equations. 

Indeed, let T^^{u!) e M be an arbitrary point in the curve r'l{uj), let [/ be a coordinate patch around 
rj'^(a;) with coordinates {x^, . . . and let x{t) — {x^{t), . . . ,x"{t)) be the expression of rj'(a;) in 

these coordinates. Then by (|2.8p . for ft, G R sufficiently small, and i G {1, . . . , n}, 

x'{to + h)- x'{to) = / {x\ h}{x{t))dt. 

J to 

Hence, by the Fundamental Theorem of Calculus, x^it) is differentiable at t^, with derivative 

x\to) = hm i {x\to + h)- x^to)) - lim ^ °^ {x\ = ft}(a;(to)), 

as required. 

The following proposition provides an equivalent expression of the Stochastic Hamilton equations in the 
Ito form (see Section [6^ . 



Proposition 2.8 The stochastic Hamilton's equations in Definition \2.2\ admit an equivalent description 
using ltd integration by using the Schwartz operator T-l(v,m) : TyV t^.M naturally associated to the 
Hamiltonian Stratonovich operator H and that can be described as follows. Let L G T„Af be a second 
order vector and f G C°° (M) arbitrary, then 

n [v, m) (L) [/] = /^J2 {/' ^'iK^y + {{/' ^j}' h.Km)e' ■ , lJ . 



Moreover, expression 12. ^|) in the ltd representation is given by 

/(r^!)-/(ro^) = ^ r {f,h,}{T^)dx= + \j2 r {{f,h,]M}{T^)d[x^,x^]. (2.10) 

We will refer to 7i as the Hamiltonian Schwartz operator associated to h. 

Proof. According to the remarks made in the Appendix 16.41 the Schwartz operator H naturally 
associated to H is constructed as follows. For any second order vector G t^M associated to the 
acceleration of a curve v (t) in V such that v (0) — v we define H {v,m) (Ly) :— Lto(o) G Tj^M, where 
m (t) is a curve in M such that m (0) = m and m (t) = H (v (t) ,m (t)) v (t), for i in a neighborhood of 
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0. Consequently, 
■H{v,m) 



d_ 
di 



fim{t)) 



dt 



{df{m{t)),mit)) = 



dt 



{dfim{t)),H{v{t),m{t))v{t)) 



t=o 



r 

^(6^i;(0)){/,;^,}(m) + (e^«(0))(d{/,/^,}(m),m(0)) 



= 5](e^^;(0)){/,/^,}(m) + (e^«(0))^(6^^,(0)){{/,/l,},MM 

= ^ E {/' '^^■IMe' + {{/' h,}, h,}{m)e' ■ e\L^ . 

In order to establish (|2.10p we need to calculate H* {v, m) {d2f{m)) for a second order form d2f{m) e 
T^M at TO e M, f e C°° (M). Since 7i* {v, to) {d2f{m)) is fully characterized by its action on elements 
of the form Ly £ TyV for some curve v (t) in such that v (0) = w, we have 

{n* (v, to) (d2/(m)), L^) = (d2/(TO), H {v, to) - W (v, rn) (L^) [/] 

= ^ E {/' f'jK^y + {{/' ^.}' ■ ^'^L^ ■ 

Consequently, H* (v, to) {d2f{m)) = Y.\^j=i{f, hj}{m)t^ + {{/, h^}, hi}{m)e' ■ 

Hence, if Th is the Hamiltonian semimartingale associated to h with initial condition To , r < (^'' is 
any stopping time, and / £ C°°(M), we have by (|6.5p and 



/ (r^) - / (r^) = / {d2f,dr'^)^ {n* {x,T^){d2f),dx) 

Jo Jo 
-jlf ({/' h,} (r'O e\dX) +jZ f ({{/, h,} , /ij (r") e' • 6^ dX) 

= E {/> /i.} (r") d^-' + f {{/, /^.} > (r") d [x\ x^] . I 



Proposition 2.9 (Preservation of the symplectic leaves by Hamiltonian semimartingales) In 

the setup of Definition ] 2. 21 let C be a symplectic leaf of {M,uj) and F'' a Hamiltonian semimartingale 
with initial condition To{uj) — Zq, where Zq is a random variable such that Zq(uj) G C for all lo G il. 
Then, for any stopping time t < (^'^ we have that T'^ £ C 

Proof. Expression (|2.6p shows that for any z C, the Stratonovich operator H{v, z) takes values 
in the characteristic distribution associated to the Poisson structure (M, {•, •}), that is, in the tangent 
space TC of C. Consequently, H induces another Stratonovich operator Hc(v, z) : TyV —^ T^C, v G V, 
z e £, obtained from H by restriction of its range. It is clear that if i : £ ^ M is the inclusion then 



H*c{v,z)oT:t = H*iv, 



(2.11) 



Let be the semimartingale in C that is a solution of the Stratonovich stochastic differential equation 



ST'^ = Hc{X,T'^)SX 



(2.12) 
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with initial condition Tq. We now show that T :^ i o is a sohition of 

ST = H{X,T)SX. 

The uniqueness of the solution of a stochastic differential equation will guarantee in that situation that 
r'' necessarily coincides with F, hence proving the statement. Indeed, for any a e Q{M), 

Since F^ satisfies (pJ2|) and T*i ■ a e n{C), by ((2ll|) this equals 

j{Hl{X,T'},){T*i-a),5X)^ j {H* {X,i oT'^){a),5X) = j {H*{X,T){a),5X), 
that is, 6T = H{X,T)5X, as required. ■ 

Proposition 2.10 (The stochastic Hamilton equations in Darboux-Weinstein coordinates) 

Let (Af, {•, •}) he a Poisson manifold and F'' he a solution of the Hamilton equations S2.5]) with initial 
condition xq G M . There exists an open neighhorhood U of xq in M and a stopping time tjj such 
that T^{lo) e U , for any lo ^ Vl and any t < tu{uj). Moreover, U admits local Darboux coordinates 
(g^, . . . , (7",pi, . . . ,p„, zi, . . . , z;) in which \2. 8]) takes the form 

p.(f^)-p,(fS) 

z.(r^) - z,(Fo^) 

where {•, - jr is the transverse Poisson structure of {AI, {•, •}) at xq- 

Proof. Let U be an open neighborhood of xo in AI for which Darboux coordinates can be chosen. 
Define tu = inft>o{Fj' G W} {tjj is the exit time of U). It is a standard fact in the theory of stochastic 
processes that tu is a stopping time. The proposition follows by writing ()2.8|) for the Darboux-Weinstein 
coordinate functions {q^, . . . ,q",pi, . . . ,pn, zi, . . . , zi). ■ 

Let C : M X ^ [0, oo] be the map such that, for any z G M, ( (z) is the maximal stopping time 
associated to the solution of the stochastic Hamilton equations (12. Sp with initial condition Tq — z a.s.. 
Let F be the flow of ()2.5p . that is, for any z G M, F (z) : [0, C (z)] ^ M is the solution semimartingale 
of (|2.5p with initial condition z. The map z e M i — > Ft{z,uj) G M is a local diffeomorphism of M, 
for each t > and almost all ui G ft in which this map is defined (see [IW89| ). In the following result, 
we show that, in the symplectic context, Hamiltonian flows preserve the symplectic form and hence the 
associated volume form 6 = w A .". A cj. This has already been shown for Hamiltonian diffusions (see 
Example [XT]) by Bismut fBSTI. 

Theorem 2.11 (Stochastic Liouville's Theorem) Let {M,lu) he a symplectic manifold, X : M+ x 
57 — > V* a semimartingale, and h : M ^ V* a Hamiltonian function. Let F he the associated Hamilto- 
nian flow. Then, for any z d M and any {t,if} G [0, C(z)], 




Ft (z,77)w = uj. 
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Proof. By [KSTl Theorem 3.3] (see also |W80) 1, given an arbitrary form a € ft'' (M) and z e M, the 

process F (z)* a satisfies the following stochastic differential equation: 




In particular, if a — a; then £x,^ .^ = for any j G {1, ...,r}, and hence the result follows. I 
2.2 Conserved quantities and stability 

Conservation laws in Hamiltonian mechanics are extremely important since they make easier the inte- 
gration of the systems that have them and, in some instances, provide qualitative information about 
the dynamics. A particular case of this is their use in concluding the nonlinear stability of certain 
equilibrium solutions using Dirichlet type criteria that we will generalize to the stochastic setup using 
the following definitions. 

Definition 2.12 A function f e C°° (M) is said to be a strongly (respectively, weakly) conserved 

quantity of the stochastic Hamiltonian system associated to h : M ^ V* if for any solution F'' of the 
stochastic Hamilton equations 112. 5\} we have that f (F'') = / (Fq) (respectively, E[f (F^)] = E[f (Fq)], 
for any stopping time t). 

Notice that strongly conserved quantities are obviously weakly conserved and that the two definitions 
coincide for deterministic systems with the standard definition of conserved quantity. The following 
result provides in the stochastic setup an analogue of the classical characterization of the conserved 
quantities in terms of Poisson involution properties. 

Proposition 2.13 Let {M, {•, •}) be a Poisson manifold, X : R+ x ft V a semimartingale that takes 
values on the vector space V such that Xq = 0, and h : M V* and f e C°°{M) two smooth functions. 
If {f,hj} = for every component hj of h then f is a strongly conserved quantity of the stochastic 
Hamilton equations i2.5\) . 

Conversely, suppose that the semimartingale X ~ X]j=i -^^^j such that ^X^,X^ = ifi ^ j. If f 
is a strongly conserved quantity then {/, hj} = 0, for any j e {1, r} such that [X^ , X^'\ is an strictly 
increasing process at 0. The last condition means that there exists A G J- and 6 > with P{A) > .such 
that for any t < S and lu G A we have [X^ , X^ti^j) > [X^ , X^]o{io), for all j £ {1, r}. 

Proof. Let F'' be the Hamiltonian semimartingale associated to h with initial condition Fq. As we 
saw in (PTTU)) . 




(2.13) 



If {/, hj} ~ for every component hj of h then all the integrals in the previous expression vanish and 
therefore / (F'') — f (Fq) which implies that / is a strongly conserved quantity of the Hamiltonian 
stochastic equations associated to h. 

Conversely, suppose now that / is a strongly conserved quantity. This implies that for any initial 
condition Fq, the semimartingale / (F'*) is actually time independent and hence of finite variation. 
Equivalently, the (unique) decomposition of / (F'') into two processes, one of finite variation plus a 
local martingale, only has the first term. In order to isolate the local martingale term of / (F'') recall 
first that the quadratic variations [X*, X^] have finite variation and that the integral with respect to 
a finite variation process has finite variation (see |LeG97[ Proposition 4.3]). Consequently, the last 



Lazaro and Ortega: Stochastic Hamiltonian dynamical systems 



10 



summand in (|2.13p has finite variation. As to the second summand, let and , j = 1, . . . ,r, local 
martingales and finite variation processes, respectively, such that = A> + . Then, 



{/, h,} (r'^) dx= - j {/, h,} {T^) dM= + j {/, h,} (r'^) dA^. 

Given that for each j, J {/, hj} (F'*) dA^ is a finite variation process and J {/, hj} (F'') dM^ is a local 
martingale (see [P90l Theorem 29, page 128]) we conclude that Z / {/. ^jl (r'*) is the 

local martingale term of / (F'') and hence equal to zero. 

We notice now that any continuous local martingale Z : M-|_ x il ^ M is also a local (ri)-martingale. 



Z^ 



< 



Indeed, consider the sequence of stopping times r" = {init > \ \Zt\ — n}, n G N. Then E 

E [n?] = r?, for aU t e R+. Hence, Z^" G I? (17) for any n. In addition, E £^ [[^^"' ^"""IJ 

(see |P90| Corollary 3, page 73]). On the other hand by Proposition 15. 5|. 



Thus, by [P90l Theorem 29, page 75] and the hypothesis {X\ X^\ = if i ^ j, 

2 



E 



z^ 



^ i? / l[o,,.] {/, h,-\ (F'O dip, / l[o,,.] {/, /i.} (F") dM' 



l[o,r"] ({/, /bl {/, h,)) (F'') d [M^ 

l[0,r"]({/,/ij}{/,/i.}) (F'')d[X^X^] 
l[0,r"] (F'')d[X-'',X^] 



= E^ 

T 

= E^ 

r 

-E^ 

i=i 

Since [X-', X-'] is an increasing process of finite variation then J Ijo^^"] {/> ^j}^ (r'') d [X-', X^] is a 
Riemann-Stieltjes integral and hence for any cj e £7 



l[o,."] (r") d {X\X^\\^ (lo) = J l[o,..(.)] (F"(c.)) d([X^X■'"] (c)) 

As [X-', AT-'] (uj) is an increasing function of f G M+, then for any j £ {1, . . . , r} 



I l[o,..] {r'^)d[x^,x^] 



> 0. 



(2.14) 



Additionally, since E 
equality. Hence, 



(Z"^")^ — 0, we necessarily have that the inequality in p.l4p is actually an 



ri[o..„]{/,/i,}2 (F'')d[X^X-'"] =0. 
Jo 



(2.15) 



Suppose now that [X^,X^] is strictly increasing at for a particular j. Hence, there exists A G 
with P{A) > 0, and J > such that [X\X^]t{uj) > [X^,X^]o{uj) for any t < 5. Take now a 
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fixed uj £ A. Since r" — > oo a.s., we can tak;e n large enough to ensure tiiat r" (w) > i, wtiere 
t S [0, 5). Tiius, we may suppose that l[o,r"] ^) = 1- As [X-', X^] (cj) is an strictly increasing process 
at zero {/, h,f {V^ (w)) d [X^ , X^] (w) > unless {/, h^f {V^ {uj)) = in a neighborhood [0, ^L) of 
contained in [0, 5). In principle 6^ > Q might depend on a; G A, so the values of t G [0, 5) for which 
{/, hjY (rj* (w)) = for any uj e A are those verifying < t < ini^^A S^j- In any case (12.15^ allows us 
to conclude that {/, hjf (rg (w)) = for any uj G A. Finally, consider any T solution to the Stochastic 
Hamilton equations with constant initial condition Fq = m G M an arbitrary point. Then, for any 
UJ E A, 

0^{f,h,f (F(JH) }2 (m). 

Since m S M is arbitrary we can conclude that {f,hj} — 0. ■ 

We now use the conserved quantities of a system in order to formulate sufficient Dirichlet type 
stability criteria. Even though the statements that follow are enounced for processes that are not 
necessarily Hamiltonian, it is for these systems that the criteria are potentially most useful. We start 
by spelling out the kind of nonlinear stability that we are after. 

Definition 2.14 Let M he a manifold and let 

6T = e{X,T)5X (2.16) 

he a Stratonovich stochastic differential equation whose solutions F : R x J7 — ^ A/ take values on M. 
Given x E M and s G M, denote hy F*'^ the unique solution of h2.16\) such that F|'^(ti)) — x, for all 
w G il. Suppose that the point zq E M is an equilibrium of 12.16\) . that is, the constant process 
Ff (w) :— zq, for all t GM. and uj G fl, is a solution of 112.1 6]) . Then we say that the equilibrium zq is 

(i) Almost surely (Lyapunov) stable when for any open neighborhood U of zq there exists another 

neighborhood V CZ U of zq such that for any z E V we have F*^'^ C U , a.s. 

(ii) Stable in probability. For any s > and e > 

lim P \snpd{Tt'^,zo) > e I = 0, 

^^^0 [t>s ) 

where d : AI x M —> is any distance function that generates the manifold topology of AI . 

Theorem 2.15 (Stochastic Dirichlet's Criterion) Suppose that we are in the setup of the previ- 
ous definition and assume that there exists a function f G C°°(M) such that d/(zo) = and that the 
quadratic form d^f(zo) is (positive or negative) definite. If f is a strongly (respectively, weakly) con- 
served quantity for the solutions of i2.16]) then the equilibrium zq is almost surely stable (respectively, 
stable in probability). 

Proof. Since the stability of the equilibrium zq is a local statement, we can work in a chart of M 
around zq with coordinates (xi, . . . ,Xn) in which zq is modeled by the origin. Moreover, using the Morse 
lemma and the hypotheses on the function /, and assuming without loss of generality that /(zq) = 0; 
we choose the coordinates {xi, . . . , Xn) so that f{xi, . . . , a;„) — Xi + ■ ■ ■ + x^. Hence, in the definition of 
stability in probability, we can use the distance function d{x, zq) = f{x). 

Suppose now that / is a strongly conserved quantity and let U be an open neighborhood of zq. Let 
r > be such that V := /^^([0,r]) C U. Let z eV with f{z) = r' . As / is a strongly conserved 
quantity f{T'^'^) = r' < r and hence F°'^ C U, as required. 

In order to study the case in which / is a weakly conserved quantity, let e > and let Ue be the 
ball of radius e around zq. Then, for any x E and s E M+, let tu^ be the first exit time of F'*'^ with 
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respect to 17^- Notice first that ii uj E ft belongs to the set {lu G fl \ supq<^<j d{Tl'^, zq) > e} ^ {uj E 
I supo<s<t / (r*'^) > e^}, then tu^{uj) < t and hence the stopped process (r^'^)^"' satisfies that 



for those values of uj. This ensures that 

e^l{^Gf^|supo<^<^rf(^=■^^o)>e} ^ f ii^''''')t"' ) ■ 

Taking expectations in both sides of this inequality we obtain 

P[ sup d Fj' , zq > e < . 

Since by hypothesis / is a weakly conserved quantity, we can rewrite the right hand side of this inequality 
as 

E[f {{^''^1"')] _ ^ {^r'lAt)] _ E[f{rr)] _ fix) 



and we can therefore conclude that 



P[ sup d{rr,zo)>e] (2.17) 



\0<s<t 

Taking the limit x zq in this expression and recalling that f{zo) = 0, the result follows. I 

A careful inspection of the proof that we just carried out reveals that in order for (|2.17p to hold, it 
would suffice to have E[f (Ft-)] < E[f (Fq)], for any stopping time t and any solution F, instead of the 
equality guaranteed by the weak conservation condition. This motivates the next definition. 

Definition 2.16 Suppose that we are in the setup of Definition \KT^ Let U be an open neighborhood 
of the equilibrium zq and let V : U —>■ 'M. be a continuous function. We say that V is a Lyapunov 
function for the equilibrium zq ifV{zQ) = 0, V{z) > for any z E U \ {zq}, and 

E[V (Tr)] < E[V (To)], (2.18) 

for any stopping time r and any solution F of 12.16\) . 

This definition generalizes to the stochastic context the standard notion of Lyapunov function that 
one encounters in dynamical systems theory. If (|2.16p is the stochastic differential equation associated to 
an Ito diffusion and the Lyapunov function is twice differentiable, the inequality (|2.18p can be ensured 
by requiring that A[l^](z) < 0, for any z £ U \ {zq}, where A is the infinitesimal generator of the 
diffusion, and by using Dynkin's formula. 

Theorem 2.17 (Stochastic Lyapunov's Theorem) Let zo G M be an equilibrium solution of the 
stochastic differential equation \2.16\) and let V : U ^ R be a continuous Lyapunov function for zq. 
Then zq is stable in probability. 

Proof. Let Ue be the ball of radius e around Zq and let := infa;g[/\t/^ ^ix)- Using the same 
notation as in the previous theorem we denote, for any x E and s £ M+, tjj^ as the first exit time of 
F'*'^ with respect to C/g. Using the same approach as above we notice that if cj e i7 belongs to the set 
{w e I supq<^<( d (Fj'"^, Zq) > e}, then tu^{lj) < t and hence the stopped process (Y^'^y"' satisfies 
that 



ViiT^n:"' (c.))-f(f^'-^(^)(c.)) 



> K, 



Lazaro and Ortega: Stochastic Hamiltonian dynamical systems 



13 



for those values of uj, since belongs to the boundary of Ue- This ensures that 

K;l{weO|supo<^<t d(r='",Zo)>e} - ^ ii^^''')t"' ) ■ 

Taking expectations in both sides of this inequality we obtain 



P sup ^(^'"^,20) > e < 



0<s<t 



We now use that V being a Lyapunov function satisfies (|2.18p and hence 



E[v{{r^'n7')] _ < E[viTr)] Vix) 



V 



We can therefore conclude that 

Pf sup d{Tr,zo)>e)<^. 

\Q<s<t J Ve 

Taking the limit a; ^ zq in this expression and recalling that V{zo) — 0, the result follows. I 
Remark 2.18 This theorem has been proved by Gihman |G66| and Hasminskii [Ha80) for Ito diffusions. 

3 Examples 

3.1 Stochastic perturbation of a Hamiltonian mechanical system and Bis- 
mut's Hamiltonian diffusions 

Let (M, {•,•}) be a Poisson manifold and hj e C°° (M), j = 0, ...,r, smooth functions. Let h : M — > 
MJ''^^ be the Hamiltonian function m 1 — *■ (hg (m) , . . . , /i^. {m)), and consider the semimartingale X : 
R+ X — i- given by 1 — > {t,Bl (w) , . . . , i3[ (w)), where , j — 1, ...,r, are r-independent 

Brownian motions. Levy's characterization of Brownian motion shows (see for instance [P90[ Theorem 
40, page 87]) that [B^,B']^ = tS^\ In this setup, the equation reads 

/ (F^') - / (fS) r {/, ho} (F") dt + j^f {/, h,] {T^) SB^ (3.1) 
Jo j=i"^o 

for any / G C°° (M) . According to (|2.10p , the equivalent Ito version of this equation is 

/ (F^) - / (Fj) = r {/, ho} (v^) dt+y^r {/, h,} (F") dB= + r {{/, h,} , h,} (f^) dt. 

Jo Jo Jo 

Equation p.ip may be interpreted as a stochastic perturbation of the classical Hamilton equations 
associated to ho, that is, 

^^^it)^{f,ho}hit)). 

by the r Brownian motions B^ . These equations have been studied by Bismut in [B81| in the particular 
case in which the Poisson manifold (M, {•,•}) is just the symplectic Euclidean space R^" with the 
canonical symplectic form. He refers to these particular processes as Hamiltonian diffusions. 

If we apply Proposition [233] to the stochastic Hamiltonian system p.ip . we obtain a generalization 
to Poisson manifolds of a result originally formulated by Bismut (see [B81', Theoremes 4.1 and 4.2, page 
231]) for Hamiltonian diffusions. See also [M99] . 
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Proposition 3.1 Consider the stochastic Hamiltonian system introduced in Then f G C°° (M) 

is a conserved quantity if and only if 

{f,ho} = {f,hi} = ...^{f,hr} = 0. (3.2) 

Proof. If (|3.2p holds then / is clearly a conserved quantity by Proposition 12. 131 Conversely, notice 
that as [B^, B^] = tS'^^ , i,j G {1, . . . , r}, and X'^{t, w) = i is a finite variation process then [X*, X^] = 
for any i,j G {0,1,..., r} such that i ^ j. Consequently, by Proposition 12.131 if / is a conserved 
quantity then 

{f,hi}^ ...^{f,hr}^0. (3.3) 

Moreover, p.ip reduces to 

r {f, ho}{r'')dt^o, 

Jo 

for any Hamiltonian semimartingale P'* and any stopping time t < (^^. Suppose that {f,ho} (toq) > 
for some toq S M. By continuity there exists a compact neighborhood U of toq such that {/, ho}\jj > 0. 
Take P'' the Hamiltonian semimartingale with initial condition Pq = mo, and let ^ be the first exit time 
of U for P'*. Then, defining t := ^ A C, 

{/, ho} (P'^) dt> f min {{/, ho} (m) \meU}dt> 0, 
Jo 

which contradicts (|3.3p . Therefore, {/, ho} = also, as required. ■ 

Remark 3.2 Notice that, unlike what happens for standard deterministic Hamiltonian systems, the 
energy ho of a Hamiltonian diffusion does not need to be conserved if the other components of the 
Hamiltonian are not involution with ho- This is a general fact about stochastic Hamiltonian systems 
that makes them useful in the modeling of dissipative phenomena. We see more of this in the next 
example. 



3.2 Integrable stochastic Hamiltonian dynamical systems. 

Let (M, uj) be a 2n-dimcnsional manifold, X : x ^ V e. semimartingale, and h : M ^ V* such 
that h = with |e^,...,e''} a basis of V* . Let H be the associated Stratonovich operator 

in (USD. 

Suppose that there exists a family of functions {fr+i, •..,/«} C C°° (M) such that the n-functions 
{/i := hi,...,fr := hr, fr+i, fn} C C°° (M) are in Poisson involution, that is, {fi, fj} = 0, for any 
i,j G {l,...,n}. Moreover, assume that F :— {fi, fn) satisfies the hypotheses of the Liouville- Arnold 
Theorem |Ar89j : F has compact and connected fibers and its components are independent. In this 
setup, we will say that the stochastic Hamiltonian dynamical system associated to H is integrable. 

As it was already the case for standard (Liouville-Arnold) integrable systems, there is a symplec- 
tomorphism that takes (M, uj) to (T" x K", J^i^i A dJ,) and for which F = F {h, /„). In par- 
ticular, in the action-angle coordinates (/i, ...,/„, 61^, 6*") , hj = hj (/i,...,/„) with j e {l,...,r}. In 
other words, the components of the Hamiltonian function depend only on the actions I :— (/i, ...,/„). 
Therefore, for any random variable Pq and any i £ {1, ...,n} 

h (r) - /. (Po) - E / {luh, (I)} (P) 6X^ = (3.4a) 
9^ (P) - 9^ (Po) = ^ / {9\h, (I)} (P) 5X^ = ^ I ^ (P) 5XK (3.4b) 
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Consequently, the tori determined by fixing I = constant are left invariant by the stochastic fiow 
associated to (13.41) . In particular, as the paths of the solutions are contained in compact sets, the 
stochastic flow is defined for any time and the fiow is complete. Moreover, the restriction of this 
stochastic differential equation to the torus given by say, Iq, yields the solution 

r 

e^iT)-9^iro) = Y.u;,ilo)X^, (3.5) 

where luj (Iq) (Iq) and where we have assumed that Xq — 0. Expression (13. 5|) clearly resembles 

the integration that can be carried out for deterministic integrable systems. 

Additionally, the Haar measure d9^ A ... A d9" on each invariant torus is left invariant by the 
stochastic fiow (see Theorem 12.111 and jL06) ). Therefore, if we can ensure that there exists a unique 
invariant measure ^ (for instance, if (|3.5p defines a non-degenerate diffusion on the torus T", the invariant 
measure is unique up to a multiplicative constant by the compactness of T" (see jIW89| Proposition 
4.5])) then ^ coincides necessarily with the Haar measure. 

3.3 The Langevin equation and viscous damping 

Hamiltonian stochastic differential equations can be used to model dissipation phenomena. The simplest 
example in this context is the damping force experienced by a particle in motion in a viscous fiuid. This 
dissipative phenomenon is usually modeled using a force in Newton's second law that depends linearly 
on the velocity of the particle (see for instance |LL76| §25]). The standard microscopic description of 
this motion is carried out using the Langevin stochastic differential equation (also called the Orstein- 
Uhlenbeck equation) that says that the velocity q{t) of the particle with mass m is a stochastic process 
that solves the stochastic differential equation 

mdq{t) = -\q{t)dt + bdBt, (3.6) 

where A > is the damping coefficient, b is a constant, and Bt is a Brownian motion. A common physical 
interpretation for this equation (see [CH06j ) is that the Brownian motion models random instantaneous 
bursts of momentum that are added to the particle by collision with lighter particles, while the mean 
effect of the collisions is the slowing down of the particle. This fact is mathematically described by 
saying that the expected value :— 'E\q\ of the process q determined by (|3.6p satisfies the ordinary 
differential equation g'e = —Age- Even though this description is accurate it is not fully satisfactory given 
that it does not provide any information about the mechanism that links the presence of the Brownian 
perturbation to the emergence of damping in the equation. In order for the physical explanation to be 
complete, a relation between the coefficients b and A should be provided in such a way that the damping 
vanishes when the Brownian collisions disappear, that is, A = when 6 = 0. 

We now show that the motion of a particle of mass m in one dimension subjected to viscous damp- 
ing with coefficient A and to a harmonic potential with Hooke constant fc is a Hamiltonian stochastic 
differential equation. More explicitly, we will give a stochastic Hamiltonian system such that the ex- 
pected value qe of its solution semimartingales satisfies the ordinary differential equation of the damped 
harmonic oscillator, that is, 

mqe(t) = -Xqe{t) - kqe{t). 

This description provides a mathematical mechanism by which the stochastic perturbations in the system 
generate an average damping. 

Consider with its canonical symplcctic form and let X : R+ x $7 — > R be the real semimartingale 
given by Xt (ui) — {t + vBt {lo)) with e R and Bt a Brownian motion. Let now /i : R^ — > M be the 
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energy of a harmonic oscillator, that is, h {q,p) := -^P^ + ^P'f'- By (2.10), the solution scmimartingales 
r'' of the Hamiltonian stochastic equations associated to h and X satisfy 

<i (r") - q {T'i) -^J{2p {T'D - .^pq {r1)) dt + ^Jp {r1) dB^ (3.7) 
p (r") - p (r[?) = / (^'p (rj') + 2mq (rj^)) dt-upjq {v'l) dB,. (3.8) 

Given that E [J p (rj') dBt] = E [J q (r?) dBt] = 0, if we denote 

q,{t):=E[q{T'l)], p, {t) -.^ E [p {vl)] , 

Fubini's Theorem guarantees that 

1 iP' 
'^'^ TO^" '-^■^ " 2m Pe (i) = -^Pe (0 - P9e (i) • (3.9) 

From the first of these equations we obtain that 

, , . v^p 

Pe (t) = mqe + -r-9e 



whose time derivative is 



2 



Pe (i) = mq^ + —qe- 



2 

These two equations substituted in the second equation of (|4.8p yield 

mq, {t) = -v'pq, it) - p + 1^ q, (t) , (3.10) 

that is, the expected value of the position of the Hamiltonian semimartingale associated to h and X 
satisfies the differential equation of a damped harmonic oscillator (|5.8[) with constants 

A — p and k = p \ V 1 

\ 4™ 

Notice that the dependence of the damping and elastic constants on the coefficients of the system is 
physically reasonable. For instance, we see that the more intense the stochastic perturbation is, that 
is, the higher v is, the stronger the damping becomes (A — p increases). In particular, if there is no 
stochastic perturbation, that is, if = 0, then the damping vanishes, k = p and (|4.10p becomes the 
differential equation of a free harmonic oscillator of mass m and elastic constant p. 

The stability of the resting solution. It is easy to see that the constant process Tt{uj) = (0,0), for 
all i G M and w G is an equilibrium solution of p.7p and (|3.8p . One can show using the stochastic 
Dirichlet's criterion (Theorem I2.15P that this equilibrium is almost surely Lyapunov stable since the 
Hamiltonian function h is a, strongly conserved quantity (by (|2.8p ) that exhibits a critical point at the 
origin with definite Hessian. 

The Langevin equation. In the previous paragraphs we succeeded in providing a microscopic Hamil- 
tonian description of the harmonic oscillator subjected to Brownian perturbations whose macroscopic 
counterpart via expectations yields the equations of the damped harmonic oscillator. In view of this, 
is such a stochastic Hamiltonian description available for the pure Langevin equation (13. 6p ? The an- 
swer is no. More specifically, it can be easily shown (proceed by contradiction) that (|3.6p cannot 
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be written as a stochastic Hamiltonian differential equation on with its canonical symplectic form 
with a noise semimartingale of the form Xt{LL!) = {fQ{t, Bt), fi(t, Bt)) and a Hamiltonian function 
h{q,p) — {ho{q,p),hi{q,p)), fo, fi,ha,hi G C°°(M). Nevertheless, if we put aside for a moment the 
stochastic Hamiltonian category and we use Ito integration, the Langevin equation can still be written 
in phase space, that is, 

dqt — Vtdt, dvt = —Xvtdt + bdBt, (3-11) 

as a stochastic perturbation of a deterministic system, namely, a free particle whose evolution is given 
by the differential equations 

dqt — Vtdt and dvt = 0. (3-12) 

Let \y} , u^} be global coordinates on associated to the canonical basis {ei, 62} and consider the global 
basis {^2"', ^2"* • d2U^]^ -^^ ^ of t*M^. Define a dual Schwartz operator S* (cc, (g, v)) : t^^ ^jM^ — > t*R^ 
characterized by the relations 

d2q I — vd2U^, d2V I — > bd2U^ — Au {d2u'^ ■ ^2"^) , 

where (q, v) e is an arbitrary point in phase space and x G M^. li X : R+ x 17 ^ R^ is such that 
X {t,uj) — {t,bBt (w)), for any {t,uj) G R+ x ft, it is immediate to see that the Ito equations associated 
to S* and X are p. lip . Moreover, if we set 6 = 0, that is, we switch off the Brownian perturbation 
then we recover p.l2p . as required. 

3.4 Brownian motions on manifolds 

The mathematical formulation of Brownian motions (or Wiener processes) on manifolds has been the 
subject of much research and it is a central topic in the study of stochastic processes on manifolds 
(see |IW89[ Chapter 5], |E89[ Chapter V], and references therein for a good general review of this 
subject). 

In the following paragraphs we show that Brownian motions can be defined in a particularly simple 
way using the stochastic Hamilton equations introduced in Dcfinition l2.2l More specifically we will show 
that Brownian motions on manifolds can be obtained as the projections onto the base space of very 
simple Hamiltonian stochastic semimartingales defined on the cotangent bundle of the manifold or of 
its orthonormal frame bundle, depending on the availability or not of a parallelization for the manifold 
in question. 

We will first present the case in which the manifold in question is parallelizable or, equivalently, 
when the coframe bundle on the manifold admits a global section, for the construction is particularly 
simple in this situation. The parallelizability hypothesis is verified by many important examples. For 
instance, any Lie group is parallelizable; the spheres S^, S^, and S"^ are parallelizable too. At the end 
of the section we describe the general case. 

The notion of manifold valued Brownian motion that we will use is the following. A M-valued 
process F is called a Brownian motion on {M,g), with g a Riemannian metric on M, whenever F is 
continuous and adapted and for every / £ C°°(M) 

/(F) - /(Fo) - i 1 AM/(F)di 

is a local martingale. We recall that the Laplacian Am (/) is defined as Am (/) = Tr(Hess/), for any 
/ G C°° (Af), where Hess / := V(V/), with V : X(M) x X(M) ^ X(M), the Levi-Civita connection of 
g. Hess / is a symmetric (0, 2)-tensor such that for any X,Y E X(M), 



Hess /(X, Y)^X [5(grad /, Y)] ~ ^(grad /, VxY). 



(3.13) 
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Brownian motions on parallelizable manifolds. Suppose that the n-dimensional manifold (M, g) is 
parallelizable and let {Yi, y„} be a family of vector fields such that for each m € M, {Yi(to), Yn{m)} 
forms a basis of T^M (a parallelization). Applying the Gram-Schmidt orthonormalization procedure 
if necessary, we may suppose that this parallelization is orthonormal, that is, g(Yi,Yj) — Sij, for any 
ij = 1, ...,n. 

Using this structure we are going to construct a stochastic Hamiltonian system on the cotangent bun- 
dle T*M of M, endowed with its canonical symplectic structure, and we will show that the projection of 
the solution semimartingales of this system onto M are M-valued Brownian motions in the sense spec- 
ified above. Let X : R+ x ^ R"+i be the semimartingale given by X{t, lu) := {t, B]{uj), . . . , B^{lo)), 
where , j = 1, n, are n-independent Brownian motions and let h = {Hq, hi, ... , hn) : T*M R"+^ 
be the function whose components are given by 

ho : T*M — > R : T*M — > R , 

OLm I -' I]"=l(am, (Vy^.y,-) (m)} Um I — > (am,Y,(m)}. 

We will now study the projection onto M of Hamiltonian semimartingales that have X as stochastic 
component and h as Hamiltonian function and will prove that they are M-valued Brownian motions. 
In order to do so we will be particularly interested in the projectable functions f of T*M, that is, 
the functions / e C°°{T*M) that can be written as f ^Jon with / e C°°(Af ) and tt : T*M M the 
canonical projection. 

We start by proving that for any projectable function / = / o tt e C°° {T* M) 



{/,'>o) = 9|g"el/,-2>,Vy,ri I and {/,/.,}=., (grad /, F,) , (3.15) 

and where {■, •} is the Poisson bracket associated to the canonical symplectic form on T*M. Indeed, let 
U a Darboux patch for T*M with associated coordinates (g^, . . . , q^^,pi, . . . ,Pn) such that {q^,pj} = Sj. 
There exists functions fj G C°°{tt{U)), with fc, j G {1, . . . , n} such that the vector fields may be locally 
written as = X^Li fj-^- Moreover, hj {q,p) = J2k=i fj i<l)Pk and 

k=l ) k=l k,i=l ^ k,i=l ^ 

= Yj [/] ° TT = 5 (grad/, Yj) o tt, 

as required. The first equality in ()3.15p is proved analogously. Notice that the formula that we just 
proved shows that if / is projectable then so is {/, hj}, with j G {1, . . . ,n}. Hence, using ()3.15|) again 
and ()3.13|) we obtain that 

{{/, hj} , h,} = Yj [g (grad/, F,)] o tt = Hess7(y„ F,) o n + g (grad/, Vy^F,) o n, (3.16) 

for j G {l,...,n}. Now, using (|3.15p and (|3.16p in (|2.10p we have shown that for any projectable 
function / = / o tt, the Hamiltonian semimartingale F'' satisfies that 

7o^(r")-7o7r(r(}) =^ / g(grad7,y,) {7roT^)dBl + -Y, / llessj (Y^Y,) {n oV'^) dt, (3.17) 
or equivalently 

Jon{T'^)-Jon{r'^)-^ I AM{7){^or'^)dt^J2j 5(grad7,y,)(^or'0dB^ (3.18) 
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Since Y^^=i 1 9 (s^'^d/, Yj) (F )dB^ is a local martingale (see |P90[ Theorem 20, page 63]), 7r(r^) is a 
Brownian motion. 

Brownian motions on Lie groups. Let now G be a (finite dimensional) Lie group with Lie algebra 
Q and assume that G admits a bi- invariant metric g, for example when G is Abelian or compact. 
This metric induces a pairing in g invariant with respect to the adjoint representation of G on g. Let 
{Ci: ■ ■ • J Cn} be an orthonormal basis of g with respect to this invariant pairing and let {vi, . . . , be 
the corresponding dual basis of g*. The infinitesimal generator vector fields {Cid ■ ■ • ,£,nG} defined by 
^icih) = TeLh ■ ^, with Lh ■ G ^ G the left translation map, /i G G, i G {1, . . . n}, are obviously an 
orthonormal parallelization of G, that is g{^iGj£,jG) ■= ^ij- Since g is bi-invariant then = ^i^j^li 

for any X,Y e X(G) (see |083[ Proposition 9, page 304]), and hence V^^^^ic — 0. Therefore, in this 
particular case the first component hg of the Hamiltonian function introduced in p.l4p is zero and we 
can hence take he = {hi, /i„) and Xa — {B}, i?") when we consider the Hamilton equations that 
define the Brownian motion with respect to g. 

As a special case of the previous construction that serves as a particularly simple illustration, we 
are going to explicitly build the Brownian motion on a circle. Let = {e*^ | 61 G R} be the 
unit circle. The stochastic Hamiltonian differential equation for the semimartingale P'' associated to 
X : M+ X ^ R, given by Xt{uj) := St(w), and the Hamiltonian function h : TS'^ ~ S"! x R ^ R 
given by h(e^^ , A) := A, is simply obtained by writing (|3.17p down for the functions /i(e'^) := cos9 and 
/2(e'®) := sin6 which provide us with the equations for the projections X'^ and F'* of P'' onto the OX 
and OY axes, respectively. A straightforward computation yields 

dX'' = -Y'^dB - \x^'dt and dY'' = X'^dB - ^Y^dt, (3.19) 

which, incidentally, coincides with the equations proposed in expression (5.1.13) of |Ok03j . A solution 
of (i3T9)l is {Xj},Yt^) = (cos St, sin Bt), that is, = e^^K 

Brownian motions on arbitrary manifolds. Let {M,g) be a not necessarily parallelizable Rie- 
mannian manifold. In this case we will reproduce the same strategy as in the previous paragraphs 
but replacing the cotangent bundle of the manifold by the cotangent bundle of its orthonormal frame 
bundle. 

Let Ox (M) be the set of orthonormal frames for the tangent space T^M. The orthonormal frame 
bundle C (M) = [j^^j^^ Ox (M) has a natural smooth manifold structure of dimension n(n+l)/2. 
We denote by tt : O (M) M the canonical projection. We recall that a curve 7 : (— e,e) C R ^ 
O (M) is called horizontal if 7* is the parallel transport of 70 along the projection 7r(7t). The set of 
tangent vectors of horizontal curves that contain a point u G O (M) defines the horizontal subspace 
HuO (M) C TuO (M) , with dimension n. The projection tt : O (M) M induces an isomorphism 
r„7r : HuO (M) Tt^^^^^M. On the orthonormal frame bundle, we have n horizontal vector fields 
Yi, i = l,...,n, defined as follows. For each u S 0{M), let Yi (u) be the unique horizontal vector in 
HuO {M) such that TuTt (Yi) = Ui, where Ui is the ith unit vector of the orthonormal frame u. Now, 
given a smooth function F G C°° {O (M)), the operator 

n 

Ao(M) (^^) = ^r,[y,[F]] 

1=1 

is called Bochner's horizontal Laplacian on 0{M). At the same time, we recall that the Laplacian 
Am (/), for any / G G°° (M), is defined as Am (./) = Tr (Hess /). These two Laplacians are related by 
the relation 

Ao(Af) (^V) = AAf (/) , (3.20) 

for any / G G°° (M) (see [H02] ). 
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The Eells-Elworthy-Malliavin construction of Brownian motion can be summarized as follows. Con- 
sider the following stochastic differential equation on O [M) (see |IW89) ): 

n 

5C/t -E^^^^*)"^^* (3.21) 

i=l 

where , j = 1, are n- independent Brownian motions. Using the conventions introduced in the 
appendix 16.41 the expression (|3.2ip is the Stratonovich stochastic differential equation associated to the 
Stratonovich operator: 

e{v,u): T„M" — > T,,0 (M) 

= ELi ^^'e» ^ Er=i v'Y, {u) , 

where {ei, . . . ,e„} is a fixed basis for R". A solution of the stochastic differential equation (|3.2ip is 
called a horizontal Brownian motion on O (M) since, by the Ito formula, 

F{U)-F (Uo) f^^ (f^^) ^^l=f2 f (^^) "^^s + l f ^o(M}iF) (Us) ds, 

4=1 •' 1=1 '' 

for any F G (O (M)). In particular, if = tt* (/) for some f e C°° [M) , by 

f{X)-f (Xo) = E / (/)] (^«) ^^''^ + 11 ^'"f (^^) 

where Xt — ir (Ut), which implies precisely that Xt is a Brownian motion on M. 

In order to generate ()3.21|) as a Hamilton equation, we introduce the functions hi : T*0 (M) R, 
1 = 1, n, given by hi (a) = {a, Yi). Recall that T*0 (M) being a cotangent bundle it has a canonical 
symplectic structure. Mimicking the computations carried out in the parallelizable case it can be 
seen that the Hamiltonian vector field X^^ coincides with Yi when acting on functions of the form 
F o TTT'oiM), where F e C°° {O {M j) and iTT'OiM) is the canonical projection ttt'O(m) ■ T*0 {M) 
O (M). By (|2.8|) . the Hamiltonian semimartingale associated to ft, = {hi, /i„) and to the stochastic 
Hamiltonian equations on T*0{M) with stochastic component X — (B}, ...,Bl^) is such that 

F o TiT'OiM) (r'*) - F o 7rT*o{M) (rp) 



T.J ^T'OiM),h,} {r':) 5Bl^T.J [^] (^T.o(Af) {r':)) sbi 



for any F £ C°°{0{M)). This expression obviously implies that U''' = nT-o{M) (r'*) is a solution 
of and consequently X'' = tt ([/'') is a Brownian motion on M. 

3.5 The inverted pendulum with stochastically vibrating suspension point 

The equation of motion for small angles of a damped inverted unit mass pendulum of length I with a 
vertically vibrating suspension point is 

y + y)'/'-A0, (3.22) 

where cj) is the angle that measures the separation of the pendulum from the vertical upright position, 
y — y (t) is the height of the suspension point (externally controlled), A is the friction coefficient, and g 
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is the gravity constant. By construction, the point {(j), </>) = (0, 0) corresponds to the upright equiUbrium 
position. It can be shown that if the function y(t) is of the form y{t) = az{u>t), with z periodic, the 
ampHtude a is sufficiently small, and the frequency w is sufficiently high, then this equilibrium becomes 
nonlinearly stable. 

We now consider the case in which the external forcing of the suspension point is given by a continuous 
stochastic process i : M+ x ^ K such that is continuous and stationary. Under this assumptions, 
the equation (|3.22|) becomes the stochastic differential equation 

d(t) = <i)dt, d^ = [j(t) - dt + e^w^^dit, (3.23) 

where e := \fajl. Observe that this equation is not Hamiltonian unless the friction term —A;/) vanishes 
(A = 0), in which case one obtains a Hamiltonian stochastic system with Hamiltonian function 0) = 
— \(e^uP'4'Vf' , —^{euj(j)l)'^) and noise semimartingale Xt — {t, [z, z],z) (the symplectic form 
is obviously l'^d(f> A dcf)). 

The stability of the upright position of the stochastically forced pendulum has been studied in |O06[ 
IIOI] , and references therein. In [O06j it is assumed that the noise has the fairly strong mixing property. 
We recall that a continuous, adapted, stationary process F : M+ x — + R has the fairly strong mixing 
property if E [F^] < oo, there exists a real function c such that c (s) ds < oo, and for any t > s 

\\E [Tt - E [Tt]\mL^ <c{t^s) ||r, - E [r,]||^. , 

where ||-||^2 stands for the norm. For example, if x is the unique stationary solution with zero mean 
of the Ito equations 

dxt = Vtdt, dyt = - (xt + yt) dt + dBt, 

where Bt is a standard Brownian motion, then — i = j/^ — i has the fairly strong mixing property. 
Using this hypothesis, it can be shown |O06[ Theorem 1] that if z : M+ x — > R is a continuously 
differentiable and stationary process such that, for any t £ R+, E[zt] = 0, £' [exp (e < oo if 
£ ~ \fajl is sufficiently small, and the process has the fairly strong mixing property, then the solution 
((/i, (/)) = (0, 0) of p.23p is exponentially stable in probability, if e is sufficiently small and < E [i^] . 
Moreover, Ovseyevich shows in [O06J Section 4] that if we put A = in p.23p and we consider hence 
the inverted pendulum as a Hamiltonian system, then the equilibrium point (0, c^) = (0, 0) is unstable. 

4 Critical action principles for the stochastic Hamilton equa- 
tions 

Our goal in this section is showing that the stochastic Hamilton equations can be characterized by a 
variational principle that generalizes the one used in the classical deterministic situation. In the following 
pages we shall consider an exact symplectic manifold (M, w), that is, there exist a one-form B € O (M) 
such that w = — d^. The archetypical example of an exact symplectic manifold is the cotangent bundle 
T*Q of any manifold Q, with d the Liouville one-form. 

In the following pages we will proceed in two stages. In the first subsection we will construct a critical 
action principle based on using variations of the solution semimartingale using the flow of a vector field 
on the manifold. Even though this approach is extremely natural and mathematically very tractable it 
yields a variational principle (Theorem 14. 9p that does not fully characterize the stochastic Hamilton's 
equations. In order to obtain such a characterization one needs to use more general variations associated 
to the flows of vector fields defined on the solution semimartingale, that is, they depend on fi. This 
complicates considerably the formulation and will be treated separately in the second subsection. 
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Definition 4.1 Let {M,uj = ~d9) be an exact symplectic manifold, X : x — > V a semimartingale 
taking values on the vector space V , and h : M ^ V* a Hamiltonian function. We denote by S (M) and 
S (R) the sets of M and real-valued semimartingales, respectively. We define the stochastic action 
associated to h as the map S : S{M) iS(R) given by 

s{T)^ J {e,sr)- J (h{r),6x), 

where in the previous expression, h (F) : M-|_ x Q ^ V x V* is given by h (T) (t, uj) :— {Xt{uj), h{Tt{^)))- 



4.1 Variations involving vector fields on the phase space 

Definition 4.2 Let M be a manifold, F : S (M) — > S (M) a map, and F G 5 (A/). A local one-parameter 
group of diffeomorphisms tp : V d M x M ^ AI is said to be complete with respect to F if there exists 
e > such that fs{^) is a well-defined process for any s € (— e, e). We say that F is differentiable at 
F in the direction of a local one parameter group of diffeomorphisms ip complete with respect to T, if for 
any sequence {s„} C M such that s„ > 0, the family 

n — >oo 

X„ = ^iF{^.Ar))-FiT)) 

Sn 

converges uniformly on compacts in probability (ucp) to a process that we will denote by ^|^_g^('/'s (r)) 
and that is referred to as the directional derivative of F atV in the direction of ips . 

Remark 4.3 Note that global onc-paramctcr groups of diffeomorphisms (for instance, flows of complete 
vector fields) are complete with respect to any semimartingale. 

Let F : M+ x fl ^ M he a M-valued continuous and adapted stochastic process and A C M a set. 
We will denote by ta — inf {t > \ Tt {oj) ^ A} the first exit time of F with respect to A. We recall 
that TA is a stopping time if A is a Borel set. Additionally, let F be a semimartingale and K a compact 
set such that Fq C K. Then, any local one-parameter group of diffeomorphisms (p is complete with 
respect to the stopped process T'^'^ . Note that this conclusion could also hold for certain non-compact 
sets. 



The proof of the following proposition can be found in Section f5.ll 

Proposition 4.4 Let M be a manifold, a € (M) a one-form, and F : S (M) S (M) the map defined 
by F (T) :— J (a, ST). Then F is differentiable in all directions. Moreover, if T : R4- x —i- M is a 
continuous semimartingale, p is an arbitrary local one-parameter group of diffeomorphisms complete 
with respect to T , and Y G X{M) is the vector field associated to p, then 



d 
ds 



F(^,(F))= 
s=o ds 



{p>la,ST)= / {£Ya,5r). (4.1) 



The symbol Eya denotes the Lie derivative of a in the direction given by Y. 

Corollary 4.5 In the setup of Definition \4-.l\ let a — uj' (1") G r2(A/), with lJ' the inverse of the vector 
bundle isomorphism iJ* : T*M TM induced by lu. Let F : x fl ^ M be a continuous adapted 
semimartingale. ip an arbitrary local one-parameter group of diffeomorphisms complete with respect to 
F, and Y G X{M) the associated vector field. Then, the action S is differentiable at F in the direction 
of ip and the directional derivative is given by 



d_ 

ds 



S 

=0 



{^s (F)) = ^r) - j {dh {uj* [a)) (F) , 5X) + lye (F) - iy0 (Fq) . (4.2) 
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Proof. It is clear from Proposition 14.41 that 



{£Ye,6T) 



in ucp. The proof of that result can be easily adapted to show that ucp 



[^:h-h) (r) 



sx 



s^O 



£Yh) {T),5X 



Thus, using and a = u} {Y) e rj(M), 

s{^,{v)) = J {£Ye,sr)-J {r),sx) = J (lyd^^ + d (m) , jr) - / {dh{Y){T) ,sx) 

= - f {a,6r)+ f {diiye), ST) - f {dh{uj*{a)) {T),SX) 



d_ 

ds 



s=0 



(a, 5T) - / {Ah {uj* {a)) (T) , 5X) + (iy^?) (F) - (Fq) . 



Corollary 4.6 (Noether's theorem) In the setup of Definition \4.1\ let ip : M. x M ^ M be a one 

parameter group of diffeomorphisms andY G X{M) the associated vector field. If the action S : S (M) — > 
S (M) is invariant by ip, that is, S {ips (F)) = S (F), for any s G R, then the function iyO is a conserved 
quantity of the stochastic Hamiltonian system associated to h : AI —* V* . 

Proof. Let F'* be the Hamiltonian semimartingale associated to h with initial condition Fq. Since ips 
leaves invariant the action we have that 



d_ 

ds 



s{^s (r'O) = 



s=0 



and hence by (|4.2p we have that 



(a, 5T^) - / {Ah {lu* (a)) (f") , 6X) + We (f'') - WO (Fq) . 



As F'* is the Hamiltonian semimartingale associated to h we have that 

{a,5T'^)^ f {Ah{oj*{a)) {V'^) ,SX) 



and hence iyfl (F'') — iyfl (Fq), as required. ■ 

Remark 4.7 The hypotheses of the previous corollary can be modified by requiring, instead of the 
invariance of the action by cps, the existence of a function F e C°°(M) such that 



d_ 

ds 



S{ip, (F'')) -F(F)-F(Fo). 



s=0 



In that situation, the conserved quantity is iyO + F. 

Before we state the Critical Action Principle for the stochastic Hamilton equations we need one more 
definition. 
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Definition 4.8 Let M be a manifold and A a set. We will say that a local one parameter group of 
diffeomorphisms ip : V x M ~* M fixes A if ips (y) = y for any y G A and any s G M such that 
{s,y) G v. The corresponding vector field Y £ X{M) given by Y{m) — ^| _^ips{m) satisfies that 

Theorem 4.9 (First Critical Action Principle) Let (M, ut = —dO) be an exact symplectic mani- 
fold, X : ]R+ X D, —> V a semimartingale taking values on the vector space V such that Xq = 0, and 
h : M ^ V* a Hamiltonian function. Let toq G M be a point in M and T : M_|_ x VL ^ AI a continuous 
semimartingale such that Fq — too- Let K be a compact set that contains the point toq. // the semi- 
martingale r satisfies the stochastic Hamilton equations \2. 7p ( with initial condition Tq = toq ) up to 
time tk then for any local one-parameter group of diffeomorphisms (/3 that fixes the set {toq} U dK we 
have 

5((^,(r^-)) =0 a.s.. (4.3) 

S = J TK 



'-{ta-<oo} 



ds 



Proof. We start by emphasizing that when we write that F satisfies the stochastic Hamiltonian 
equations (|2.7p up to time tk we mean that 



{P,ST)+ J {dh{Lu*i(3))ir),6X)j =0. 
For the sake of simplicity in our notation we define the linear operator Ham : f2(M) —> iS(M) given by 



Ham ip) 



{f3, ST) + J {dh {oj* (/?)) (F) , sx)^ , pen (M) . 



Suppose now that the semimartingale F satisfies the stochastic Hamilton equations up to time tk. Let 
(fi he a local one-parameter group of diffeomorphisms that fixes {toq} U dK, and let Y G X{M) be the 
associated vector field. Then, taking a = uj'^ (Y), we have by Corollary [431 

d_ 

ds 



s=0 



SiipsiT^n)^- / {a,Sr-'<)- / {dh{u;*ia))ir^'<),SX)+iYe{T^n, (4-4) 



since Y (toq) = and hence iyd (Fq) = 0. Additionally, since F is continuous, l^.rK<oo}^TK £ ^-nd 
y\dK = 0. Hence, 



'-{tk<oo} 



d 
ds 



s=0 



'-{rA'<oo} 



(a, Sr^'^} + / {dh {lo* {a)) (F^^-) , 5X) 



Now, Proposition IS . 51 and the hypothesis on F satisfying Hamilton's equation guarantee that the previous 
expression equals 



4rK<oo} 



d_ 

ds 



s=0 



'-{tk<oo} 



'-{tk<oc} 



{a, ST^'<) + / {dh {oj* {a)) (F^^) , 5X) 



{a,5T)+ / {dh{uo*[a)) {T) ,5X) 



-l{rK<oo} [Ham(a)^^,] = a.s., 



as required. 
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Remark 4.10 The relation between the Critical Action Principle stated in Theorem 14.91 and the clas- 
sical one for Hamiltonian mechanics is not straightforward since the categories in which both are formu- 
lated are very much different; more specifically, the differentiability hypothesis imposed on the solutions 
of the deterministic principle is not a reasonable assumption in the stochastic context and this has se- 
rious consequences. For example, unlike the situation encountered in classical mechanics, Theorem 14.91 
does not admit a converse within the set of hypotheses in which it is formulated. 

In order to elaborate a little bit more on this question let (M, oj = —dO) be an exact symplectic 
manifold, take the Hamiltonian function h e C°°(M), and consider the stochastic Hamilton equations 
with trivial stochastic component X : R+ x — > R given by Xt(aj) — t. As we saw in Remark 12.71 the 
paths of the semimartingales that solve these stochastic Hamilton equations are the smooth curves that 
integrate the standard Hamilton equations. In this situation the action reads 

^(r) = J {0,ST)- J h{rs)ds. 

If the path Tt(oj) is differentiable then the integral (J(6', (5r)) (oj) reduces to the Riemann integral 
/rt(w) ^ '5'(r)(cj) coincides with the classical action. In particular, if F is a solution of the stochastic 
Hamilton equations then the paths Ft(ti;) are necessarily differentiable (see Remark l2.7p . they satisfy the 
standard Hamilton equations, and hence make the action critical. The following elementary example 
shows that the converse is not necessarily true, that is one may have semimartingales that satisfy ()4.3p 
and that do not solve the Hamilton equations up to time tk- 

We will consider a deterministic example. Let mo, mi £ M be two points. Suppose there exists 
an integral curve 7 : [toi^i] ^ of the Hamiltonian vector field defined on some time interval 
[<o,ii] such that 7(^0) — ™o and 7(^1) = mi. Define the continuous and piecewise smooth curve 
a : [0, tl] ^ M as follows: 

mo if t G [0, to] 

-fit) ifte[to,ti]. 

Let (fi he a. local one-parameter group of diffeomorphisms that fixes {mo, mi}. Then by (|4.2p 



ait) = 



d 
ds 



S {ips (cr)) 



s=0 







/ a + 




- t 









(a, Xh) (ait)) dt + {9{a{t)),Y{ait))) - (0(mo), r(mo)). 



where Y (m) 
equations on [tg^ti 



eI.s-o '^^ ^'^^ m G M and a = uj^ (Y) 
and a (mp) = 0, it is easy to see that 

4- S (ips (ct)) 



Using that a satisfies the Hamilton 



0, 



that is, a makes the action critical. However, it does not satisfy the Hamilton equations on the interval 
[0, tl] , because they do not hold on (0, to). This shows that the converse of the statement in Theorem l4.9l 
is not necessarily true. In the following subsection we will obtain such a converse by generalizing the 
set of variations allowed in the variational principle. 



4.2 Variations involving vector fields on the solution semimartingale 

We start by spelling out the variations that we will use in order to obtain a converse to Theorem 

Definition 4.11 Let M he a manifold and F a M-valued semimartingale. Let sq > 0; we say that the 
map S : (— so. So) x H^-i- x — > M is a pathwise variation o/F whenever E° = F; for any t G R-f a.s.. 

We say that the pathwise variation T, of T converges uniformly to F whenever the following 
properties are satisfied: 
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(i) For any f e C°° (M), f (S") -> / (F) in ucp as s ^ 0. 

(ii) There exists a process Y : M-|_ x D, TM over T such that, for any f G C°° (Af ), the Stratonovich 

integral J Y [/] 6X exists for any continuous real semimartingale X ( this is for instance guaranteed 
ifY is a semimartingale) and, additionally, the increments (/ (E") — / (r))/s converge in ucp to 
Y [f] as s ^ Q. We will call such a Y the infinitesimal generator of E. 

We will say that E ( respectively, Y ) is bounded when its image lies in a compact set of M ( respectively, 
TM). 

The next proposition shows that, roughly speaking, there exist bounded pathwise variations that 
converge uniformly to a given semimartingale with prescribed bounded infinitesimal generator. 

Proposition 4.12 Let T be a continuous M-valued semimartingale T, K <Z M a compact set, and tk 
the first exit time ofT from K . Let Y : M-|_ x ^ TM he a bounded process over T'^'^ (that is, the image 
ofY lies in a compact subset ofTM ) such that J Y [f] SX exists for any continuous real semimartingale 
X and for any f € C°° (Af). Then, there exists a bounded pathwise variation S that converges uniformly 
to F'^^ whose infinitesimal generator is Y . 

Proof. Let {{Vk,(pk)} ^.fzj,} be a countable open covering of M by coordinate patches such that any 
Vk is contained in a compact set. This covering is always available by the second countability of the 
manifold and Lindelof's Lemma. Let {Uk}k^j^ be an open subcovering such that, if t/fc C for some k, 
i e N, then t/fe C V^j. Let {rmj^gpj be a sequence of stopping times (available by Lemma 3.5 in |E89| ) 
such that, a.s., tq = 0, r,„ < Tm+i, sup„ r,„ = oo, and that, on each of the sets [r„j, T„i+i]r]{Tm^i > Tm} 
the semimartingale F takes values in the open set t/fc(m)i for some kirn) G N. Since K is compact, it 
can be covered by a finite number of these open sets, i.e. K C Uj^jUkj , where | J| < oo. 

Let Xkj = [x].., ■ . ■ n = dim (Af) be a set of coordinate functions on Uk.^m) s-^d i^kj^'^kj) = 

{xl^ , . . . ,x^.,vl., . . . , ) the corresponding adapted coordinates for TM on n^lj {Uk^ (m) ) ■ Since Y is 
bounded and covers F"^^, and on [r^jTm+i] n {Tm+i > Tm} the semimartingale F takes values in the 
open set Uk^(m), there exist a Sk^ > such that, on [rm,T„i+i] n {t„i+i > T,n}, the points (x^^. (F) + 
sv^. (Y) , . . . ,x^^. (F) + sv"^. (Y)) lie in the image of some coordinate patch Vk^ containing Uk^^m) for 
aU s G {~Skj,Skj)- Let sq = min^gj {sfc^. } . Now, since the sets of the form Im ■= [Tm,Tm+i) n 
{Tm+i > T,n} C M+ X fi, m G N form a disjoint partition of R+ x fl we define E as the map that for 
any m G N satisfies 

) n {Tra+l > T„} > Vfc^. 

(S, t, UJ) I > (^jT^ {Xk^ (Ff (cj)) + SVk, (Yt (w))) . 

Observe that by construction the image of E is covered by a finite number of coordinated patches and 
therefore, by hypothesis, contained in a compact set. E is hence bounded. More specifically 

{E^ (lu) I (s, t, u) G (-so, so) X M X fi} C IJ Vk^ . (4.5) 

jeJ 

It is immediate to see that E is a pathwise variation which converges uniformly to F'^^' . Indeed, if 
/ G C°° (Af) has compact support within one of the elements in the family \Uk ] it can be easily 
checked that 

/(E^)™>/(F) and l^^^l^ISEl ^ Y [f] . (4.6) 

ucp s ucp 

If, more generally, / G C°° (Af) has not compact support contained in one of the {Ukj}^^j, observe 
that, by (|4.5p . we only need to consider the restriction of / to Ujej ^hj- Take now a partition of the 



Lazaro and Ortega: Stochastic Hamiltonian dynamical systems 



27 



unity {'/'fclfcgfj subordinated to the covering {I7fc}j,gpj. Since {supp is a locally finite family 

and UjeJ ^'^j contained in a compact set because, by hypothesis, so is each Vk^ for any j G J, then 
among all the {^fc}j.gpj only a finite number of them have their supports in {Ukj say {0fe. with 
|/| < oo. Thus, 



/k 



1=1 



<t>kj 



and since each 4>k.f is a function similar to those considered in (14. 6p it is straightforward to see that 
those implications also hold for /. ■ 

The following result generalizes Proposition 14.41 to pathwise variations of a semimartingale. The 
proof can be found in Section [5.21 



Proposition 4.13 Let T be a M-valued continuous semimartingale T , K C M a compact set, and tk 
the first exit time ofT from K. Let Y, be a bounded pathwise variation that converges uniformly to F'^^ 
and Y : x — > TM the infinitesimal generator of S that we will also assume to be bounded. Then, 
for any a ^ Q (M), 



lim — 

ucp s 



(iyda, 5V^'<) + {a (F^-) ,Y) - {a (F^-) , Y)^^^ . 



The next theorem shows that the generalization of the Critical Action Principle in Theorem 14.91 to 
pathwise variations fully characterizes the stochastic Hamilton's equations. 

Theorem 4.14 (Second Critical Action Principle) Let {M,uj = —dO) be an exact symplectic man- 
ifold, X : X Q ^ V a semimartingale that takes values in the vector space V, and h : M ^ V* a 
Hamiltonian function. Let mo be a point in M and F ; x D, ^ M a continuous adapted semimartin- 
gale defined on [0, ^r) such that Tq — mo. Let K <Z M be a compact set that contains niQ and tk the 
first exit time ofT from K. Suppose that tk < oo a.s.. Then, 

(!) For any bounded pathwise variation E with bounded infinitesimal generator Y which converges uni- 
formly to F'^^ uniformly, the action has a directional derivative that equals 



d 
ds 



SiY.") 



1 



s=0 



lim-[^(S 

+0s 



SiV'')] 

(iyrf0,5F--)- / (Y]h]{T^'<),Sx) + {eiT--),Y)-{e{T^-),Y)^^,, (4.7) 



where the symbol Y [/i](F'^^) is consistent with the notation introduced in Definition \4-. 1\ 

(ii) The semimartingale F satisfies the .stochastic Hamiltonian equations {2.1^ with initial condition 
Fq = mo up to time tx if and only if, for any bounded pathwise variation E : (— sq, sq) x R_|_ x — s- 
M with bounded infinitesimal generator which converges uniformly to F"^^' and such that Eg = mo 



and EJ 



"t-^ a.s. for any s G (-so, so) 
d 



ds 



s{j:') 



s=0 



= a.s.. 



Proof. We first show that the limit (|4.7p exist. Let E be an arbitrary bounded pathwise variation 
converging to F uniformly and Y : M+ x fl ^ TM its infinitesimal generator, that we also assume to 
be bounded. We have 



i[5(E^-)-5(r"^)] = - 

s s 



hiY.") - h(T'"'-) ,SX 



(4.8) 
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By Proposition 14. 13[ the first summand in the right hand side of (|4.8p converges ucp to 

{iyde, sr^-) + {6 (r--) ,Y)-{e (r--) , y),^^ . 



as s ^ 0. An argument similar to the one leading to Proposition 14.131 shows that the second summand 
converges to / (y[7i](r^^), Hence, 



iinii[5(E^)~5(r^-)] = j {iYde,dr^n~J {Y[h]{r^n,sx) + {e{r^-),Y)-{eir^-),Y),^,. 

If we denote by r] := —iydO = iyuj the one-form over P"^^' built using the vector field Y over P"^^, the 
previous relation may be rewritten as 



_d_ 
ds 



{V,ST^-)- / {dhir^-){u;*{7^)),SX) + {eir^-),Y)-{9{T^-),Y)^^,. (4.9) 



We are now going to prove the assertion in part (ii). Recall that the hypothesis that P satisfies the 
stochastic Hamilton equations up to time tk means that 



(/3,<5P)+ / {{dh-u;*{P)) ir),SX) 



= 0, 



(4.10) 



for any f3 € ft (M). We now show that this expression is also true if we replace f3 with any process 
T] : R+ X — > r*M over P such that the two Stratonovich integrals involved in (|4.10p are well-defined 
(for instance if /? is a semimartingale) . Indeed, invoking ( [E89[ 7.7]) and Whitney's embedding theorem, 
there exist an integer p G N such that the manifold M can be seen as an embedded submanifold of 
W. In this embedded picture, there exists a family of functions {f^, ...,/^} C C°° (R^) such that the 
one-form rj may be written as 

where the Zj : R+ x ft —>■ M., j G {1, ...jp}, are real processes. Moreover, using the properties of the 
Stratonovich integral (see [E89[ Proposition 7.4]), 



( / {V,ST)+ I {{dh-Lo*ir^)) {T),SX)) " 

I z,6(^l ((d/^^p) + 1 {{dh-c.*{df))ir),sx)'^^ 
^ J z,s(^J (^{dp,ST) + J {{dh-u;* {dr)){r),sx) 



where the last equality follows from Proposition 15.51 Pherefore, since d/^ is a deterministic one-form 
we can conclude that (/ {{df^,ST) + J {{dh ■ uj* (d/^)) (P) , JX)))^'"' = 0, which justifies why (lilB 
also holds if we replace (3 gVI (M) by an arbitrary integrable one-form rj over P. 

Suppose now that P satisfies the stochastic Hamilton equations up to tk and let E : (— so,so) x 
M_l_ X f2 ^ Af be a pathwise variation like in the statement of the theorem. We want to show that 



■ d 

ds 



— a.s.. 



s=0 
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Due to (|4.9p . we have that 



ds 



Since Eg — mo and — Tr^ a.s. for any s G (—so, so), then Yq = Yr^ = a.s. and both 

{0 (r""^) , Y)^^ and {0 (F^^) , Y)^^^ vanish. Moreover, 

{v,sr^n + J (d/i(r--)(^#(7?)),5x)^ = (^(^J (?7,^r--) + y (d/i(r-) (,7)) 

{v,Sr)+ J {dh{T) {u;* {rj)),SX)Y^ 

'(4.11) 



which is zero because of (|4.10p . In the last equality we have used Proposition [53] 

Conversely, suppose that [ ^ |g_o '•^^-'] ~ ^'^^ arbitrary bounded pathwise variations 

tending to uniformly, like in the statement. We want to show that (|4.10p holds. Since our pathwise 
variations satisfy that Iq = Ytk — ^-S-j we obtain that 



d_ 

ds 



(^j {rj, ST-'<) + J (dh (r--) {u;* (,7)) , SX)^ = (4.12) 



where rj is an arbitrary bounded one form over F. Suppose now that 77 is a semimartingale. Then 
l[o.t]77 : M+ X 57 ^ T*M is again bounded and expressions 

(l[o.t]r,,(5r^-) and f {dh{T^^) {to* ,SX) 



are well-defined by Proposition 15 . 71 because both V^^ and X are continuos semimartingales. We already 
saw in (|4.1ip that (|4.12p is equivalent to 

(^1 iv, ST) + J (dh (r) {u;* (77)) , 6X)^ = 0. 
Replacing rj by l[o^t]il (|4.12p and using again the Proposition 15.71 we write 

0=1/ (l[o,t]r,, 6r) + J {dh (r) {to* {l[o,t]v)) , 6X)^ =(^I^J (r,, ^F) + J {dh (F) {to* (,7)) , SX) 

(ry,<5F)+ J {dh (T) {u;* (rj)) , SX)^ ^ = (^(^J {fj,dT) + J {dh (T) {to* (r^)) ,5X) 

Since t is arbitrary this implies that the process (/ {ri,6r) + J {dh{r) {uj'^ {rj)^ ,5Xyj^'^ is identically 
zero, as required. ■ 

5 Proofs and auxiliary results 
5.1 Proof of Proposition 14.41 

Before proving the proposition, we recall a technical lemma dealing with the convergence of sequences 
in a metric space. 
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Lemma 5.1 Let {E,d) he a metric space. Let {x,i\^^^^ he a sequence of functions Xn ■ (0, (^) — > E 
where (0, S) d M. is an open interval of the real line. Suppose that Xn converges uniformly on (0, S) to 
a function x. Additionally, suppose that for any n, the limits lim x„ (s) — x^ ^ E exist and so does 

s—fO 

lim X*. Then 

n— »oo 

lim X (s) = lim x* . 

s — >0 n — >oo 

Proof. Let e > be an arbitrary real number. We have 

d (x (s) , lim X* ) < d{x{s),Xk (s)) + d{xk is),xl) + d (x^, lim x* ) . 

From the definition of limit and since Xk (s) converges uniformly to x on (0, S) , we can choose kg such 
that d (x^, lim„^oo x* ) < | and d (x (s) , x^ (s)) < |, simultaneously for any k > kg. Additionally, since 
limg^o Xk (s) = x^ we choose Sq small enough such that d (x^ (s) , x^) < |, for any s < sg. Thus, 

d ( X (s) , lim X* 1 < e 



(x (s) , 



n — >oo 



for any s < sq. Since e > is arbitrary, we conclude that lim x (s) ~ lim x* . T 

s — >0 n — *oo 

Proof of Proposition [474l First of all, the second equality in (|4.ip is a straightforward consequence 
of |E89[ page 93]. Now, let {Ukj^^iq be a countable open covering of M by coordinate patches. By |E89i 
Lemma 3.5] there exists a sequence {Tm}^^^^^ of stopping times such that tq = 0, < Tm+i, sup„j t„j = 
oo, a.s., and that, on each of the sets 

[Tm, Tm+l] C\ {r„ < T,„+i} := { (t , w) £ M+ X 17 | Tm+1 (lo) > Tm (uj) and t G [t,„ (uj) , Tm+1 {lo)]} (5.1) 

the semimartingale F takes its values in one of the elements of the family 

Second, the statement of the proposition is formulated in terms of Stratonovich integrals. However, 
the proof will be carried out in the context of Ito integration since we will use several times the notion 
of uniform convergence on compacts in probability (ucp) which behaves well only with respect to this 
integral. Regarding this point we recall that by the very definition of the Stratonovich integral of a 
1-form a along a semimartingale F we have that 

((^>,(5F)= / (d2 ((y9*a) , dF) and / {£Ya,5T) ^ [ (da (i^ya) , dF) . (5.2) 



The proof of the proposition follows directly from Lemma 15.11 by applying it to the sequence of 
functions given by 

{s):=( f (- [d2 i^ta) - d2 (a)] , dF^ 



This sequence lies in the space D of caglad processes endowed with the topology of the ucp convergence. 
We recall that this space is metric |P901 page 57] and hence we are in the conditions of Lemma [5. II In 
the following points we verify that the rest of the hypotheses of this result are satisfied. 

(i) The sequence of functions {x„(s)}„gN converges uniformly to 

x{s):= I /l[d2(<^:a)-d2(«)],dF 



The pointwise convergence is a consequence of part (i) in Proposition l5.6l Moreover, in the proof of that 
result we saw that if d : D x D ^ M_|_ is a distance function function associated to the ucp convergence, 
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then for any t 6 K+ and any s £ (0, e), d{xn{s),x{s)) < P({t„ < t}). Since the right hand side of this 
inequahty does not depend on s and P({t„ < t}) — > as n ^ oo, the uniform convergence foUows. 

(ii) 



linia;„(s) = f / (^2 (-^ya) , cJr) 

By the construction of the covering {C^fejfcgN stopping times {Tm}mefiJ there exists a k{m) G N 

such that the semimartingale T takes its values in Uk{m} when evaluated in the stochastic interval 

(Tn,Tn+i] C [t„,t„+i] (H {r„ < T„+i}. Now, siucc c?2 is a linear operator and ^ {{ifla) — a) (m) 
£YCf{m), for any m G A/, we have that i {d2 (<p*a) — d2a) (m) ^2 (m). Moreover, a straight- 

forward application of Taylor's theorem shows that -i {d2 {'p*a) — d2a) |t/fc(„) d2 {£YCf) \uk(„^■, uni- 
formly, using a Euclidean norm in T*U^jnj (we recall that ?7fe(m) is a coordinate patch). This fact 

immediately implies that ^^^j^j i {d2 {(PgCi) — d2a) (F) T-^^j^](i2 (•'-ya) (r) in ucp. As by con- 

struction the Ito integral behaves well when we apply it to a ucp convergent sequence of processes we 
have that 

lim J {d2 ifloi) - d2a) {T),dT^ = J (^2 (.£ya) (r),(ir) . (5.3) 

Consequently, 

1,™ I ' \ ^ ["^2 ivlct) - d2 (a)] , dr 



+0 



i [d2 ((/.» - d2 (a)] , dr\) -( j /l[d2 (ip^a) - d2 (a)] , 



s / / \ \ s 



'is i: [(/ 

n— 1 f / \ \ ''^''"1 

l^™ X! / \ ~ (^2 i(p*s(x) - d2a) , \ = ^ / 1(t„,t„.+i] (c^2 (i^ya) , rfF) 



(^2 (-Cya) , dr) 

where in the second equality we have used Proposition 15.51 and the third one follows from (15. Sp . 
(iii) 

lim a;* = / (^2 (-f y") , dT) . 

n— »oo _y 



It is a straightforward consequence of Proposition 15.61 

The equation (14.11) follows from Lemma [5.11 applied to the sequences {a^nlnsN and {a;*}„gN, and 
using the statements in (i), (ii), and (iii). ■ 

5.2 Proof of Proposition [47T3] 

We will start the proof by introducing three preparatory results. 

Lemma 5.2 Let {^ri}„gN o-nd {Yn}^^^ he two sequences of real valued processes converging in ucp 
to a couple of processes X and Y respectively. Suppose that, for any t G M+, the random variables 
sup„gN supo<s<j |(X„)^| and supo<s<j \Ys\ are bounded (their images lie in a compact set ofR). Then, 
the sequence X^Yn converges in ucp to XY as n —>■ oo. 
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Proof. We need to prove that for any e > and any t G K+, 



P ( <! sup - iXY)J <e}] ^ 1. 

^0<s<t 



n—^co 



First of all, note that 

sup \{XnYn)s - {XY)J < sup |X„| |y„ - F] + sup |y| |X„ - X| . 

0<s<t 0<s<t 0<s<t 

Hence, we have 

sup - {XY)j < 4 3 I sup |x„| |y„ - y| + sup \y\ |x„ - x| < £ 

0<s<t J Lo<s<t 0<s<t 



Denote 



D <^ sup |x„| |y„ - F| < - ^ n <^ sup |y| \Xn -x\< 

KO<s<t ^) Lo<s<t 



A„ := <j sup |X„||y„-y| < -;>, and Bn := \ \Y\\Xn - X\ < 



and let c be a constant such that sup^gj^j supQ<g<j |(^n)sl < c and supQ<g<j \ Ya\ < c, available by the 
boundedness hypothesis. Then, 

i>P(^J>pf| sup |y„-y|<±j) — . 1, 

l>P{Bn)>p(l sup \Xn-X\ , 

V lo<s<t J / n->oo 

Thus, P {An) ^ 1 and P (B„) ^ 1 as n ^ oo. But as P (A„ n B„) = P (A„) + P - P U B„), 
we conclude that 

P(A„nP„) 1. 

n— >oo 

Since A„ n B„ C {supo<g<i |(X„y„)^ - (Xy)^| < e}, we obtain 



p ( { sup |(x„y„)^ - (xy)j <s}] 1. 

.Q<s<t 



n — >oc 



Lemma 5.3 Let {Xn}^^^ be a sequence of real processes converging in ucp to a process X. Let t be a 
stopping time such that r < oo a.s.. Then, the sequence of random variables {{Xn)^.}^^^ converge in 
probability to {X)^. 

Proof. First of all we show that since r < oo a.s., then P ({r > t}) converges to zero as t ^ oo. 
By contradiction, suppose that this is not the case. Then, denoting An '■= {t > n}, wc have that 
An+i ^ An, so P (An) forms a non-increasing sequence of real numbers in the interval [0,1]. Since 
this sequence is bounded below, it must have a limit. This limit corresponds to the probability of the 
event {r = oc}. If it is strictly positive then there is a contradiction with the fact that r < oo a.s.. So 
P ({t > t}) tends to zero as t — » oo. 

We now prove the statement of the lemma. Take some e > and an auxiliary t G The set 
{|(^?i)r ~ \ > s} can be decomposed as the disjoint union of the following two events, 

({|(x„)^ - Xr\ >e}n{T< t}) y ({|(X„), - Xr\ >e}n{T> t}) . 
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The first one is contained in the set {supg<s<( \ {Xn)g — Xs\ > e} whose probabihty, by hypothesis, 
converges to zero as n — > oo. Regarding the second one, 

P{{\{X,,)^-Xr\>e}n{T>t})<Pi{T>t}). 

But P ({r > t}) can be made arbitrarily smaU by taking the auxiliary t big enough. In conclusion, for 
any e > 0, 



Pi{\{X^)^-Xr\>e}) 







in probability. 



Lemma 5.4 Let {Xn}^^jq be a sequence of real processes converging in ucp to a real process X and t 
a stopping time. Then, the stopped sequence {X^}^^^^ converges in ucp to X'^ as well. 



Proof. We just need to observe that, for any t G 



sup \iX:)^-X:\^ sup |(X„)^^^-X,^,| < sup \{X.n),-X, 

0<s<t 0<s<t 0<s<t 



and, consequently, for any £ > 0, 



0<s<t 



sup |(X„)^ -Xs\<e\c\ sup \{X:)^ - Xl\ < e 



0<s<t 



Hence, since by hypothesis P ({supq<^<( K-'fn)^ — Xs\ < e}) converges to 1 as n ^ oo, then so does 

p({supo<,<,|(x;^),-xj| <£}). 

We now proceed with the proof of the proposition. We will start by using Whitney's Embedding 
Theorem and the remarks in !E89' §7.7] to visualize M as an embedded submanifold of M.P , for some 
p G N, and to write down our Stratonovich integrals as real Stratonovich integrals. Indeed, there exists 
a family of functions {h^, h^^ C C°° {M.P) such that, in the embedded picture, the one form a can be 
written as a = Zjdh^ , where Zj £ C°° (M.^) for j £ {1, Therefore, using the properties of 

the Stratonovich integral (see ^E89i Proposition 7.4]), 



j=i 



(5.4) 



Adding and subtracting the term X]j=i / Zj (S^) 6h^ (F"^^) in the right hand side of (j5.4|) . we have 



^ 1 



Zj (s") {T,') - / Zj (E'*) 6h^ (r^^') 



^ 1 



(1) 



(5.5) 



(2) 



We are going to study the terms (1) and (2) separately. We start by considering 

<Jn = {0 = To" < Ti" < . . . < Tl < oo} , 
a sequence of random partitions that tends to the identity (in the sense of |P901 page 64]). 
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The expression (1): We want to study the ucp convergence of i [/ Zj (E^) 5h^ (S") - / Zj (E") Sh^ (F^ 
as s ^ 0. Define 



1 f'^' 1 



i=0 
k„-l , 



i=0 



which corresponds to the discretization of the Stratonovich integrals ^ [/ Zj {T,") Sh^ (S'') — J Zj (E*) Sh^ (F"^^ )] 
using the random partitions of cr„. Indeed, by JP90' Corollary 1, page 291], 



Xn (s) 



ucp s 
n— i-oo 



Zj (E") Sh^ (E") - / Zj (E") (r^^-) 



On the other hand, as T" < oo a.s. for any i G {1, ...,fc„}, part (i) in Definition 14.111 and Lemma [5?3 
imply that 

i (z, (E^)^„^^ + (E^)j,„) — . i (z, (r^-)T.;, + z, (r--)^.) 



The convergence above is in probability but, for convenience, we prefer to regard these random variables 
as trivial processes. Furthermore, part (ii) in Definition 14. 1 II and Lemma [ 



ucp 



hi {T^' f^ - {T^^ f^ _ /hi (E'^) - hi {r^'<y 



ucp 




by Definition ()4.1ip item 3 and Lemma (15. 4|) . Now, since by hypothesis E and Y are bounded then so 
are ^ (^Zj (E*),^,,, + Zj (E*)j,„^ and Y [/i-'] = iydh^ {dh^ is only evaluated on the compact K since Y 
is a vector field over F'^*-' ) and hence by Lemma 15.21 



In addition, by [P90i Corollary 1, page 291], 



ucp 

n— >oo 



Hence, by Lemma |5. 11 we conclude that 



Zj (Y.') Sh^ (E") - / Zj {Y.") Sh^ (F''^) 



ucp 



I Zjir^-)S{Y [hq). 



(5.6) 
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The expression (2): We want to study now the ucp convergence of / {Zj (E*) — Zj (F"^^)) Sh^ (F"^^) 
as s ^ 0. As in the previous paragraphs, we define 



(T^K fr+i {Y^K fi 



\ 2 = 



Ed ^ + 



as a discretization of the Stratonovich integral i J {Zj (S*) — (F"^^)) J/i-' (F"^^) using cr„. Then, by 
construction, 

Vn is) f {Z, (E^) - Z, (F--)) Jft-'- (F-) . 

ucp s J 

n — *oo 

On the other hand, invoking Definition 14.111 and Lemma lS.SI we have that 



Z,m^,. -Z,(F--), 



> Y [Zj]rp„ 

'+1 S^Q 



Z, (E^)^,. - Z, (F--)^„ _ /z, (E^) - Z, (F--) 



ucp 



Y[Z.j\ 



We now use again the boundedness of E and Y to guarantee the boundedness of Y [Zj\j,„ — (iydZj)j,„ 
and Y [Zj\j,„ = {iYdZj)j,„ (notice that dZj is only evaluated on the compact set K because F is a 
vector field over F"^^ K). Therefore, by Lemma [521 



Additionally, the sequence {x* }„gN obviously converge in ucp to J y [Zj] S {h^ (F"^^')) asn ^ oo. Hence, 
by Lemma [5TT1 we conclude that 



(Zj(E-')-Zj-(F^^))J/i^(F^^') 



ucp 
s-*0 



Y[Z,]5{hnT^-)). 



(5.7) 



To sum up, if we substitute (|5.6p and (|5.7p in (|5.5p we obtain that 



ucp 

s-0 ■J = i 



Using the integration by parts formula, 

Z, (F-- ) S {Y [h^ ) ^ (F-- ) Y [h^ - (Z, (F-- ) Y [h^ ) ,=o " / ^ ['^■'l ^ (^^^ )) 
= (a (F--) ,Y)-{a (F^-) , y),^^ - [y [h^ S [Z, (F^-)) 
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and, consequently, 

i [/ {a, - / (a, 5T^-)] J Y [Z,] 5 (F^-)) - / y [h^] 5 {Z, (F^-)) 

+ (a(r'-)V)-(a(r^-),i^)t=o- 

In order to conclude the proof, we claim that 

j Y \z,\ 6 {h^ (r--)) -Jy [h^ s {z, (r--)) = J {Wda, sr^'<) . (5.8) 

Indeed, 

(p \ p p 

ZjAW = dZj A d/i^ and iyda = J2(^ t^jl ~ ^ i^^] 

which proofs (|5.8p . as required. ■ 

5.3 Auxiliary results about integrals and stopping times 

In the following paragraphs we collect three results that are used in the paper in relation with the 
interplay between stopping times and integration limits. 

Proposition 5.5 Let X he a continuous semimartingale defined on [0, Cx) and T a continuous semi- 
martingale. Let r, ^ be two stopping times such that t < ^ < C^x ■ Then, 

(X-Tf = {l[o,r]X)-T=iX-Tn and (X • r)« - (X • = • T 

An equivalent result holds when dealing with the Stratonovich integral, namely 

XSr] = ( X5T^ =( ( X^5T 



Proof. By [P90l Theorem 12, page 60] we have that Ip.r]^ - T ^ {X - VY ^ {X ■ V). Therefore, 

{X ■ vf - {X ■ vy = i[o,5]X • r - i[o,,]X • r = [(i[o,^] - ip,.]) x]-t = • r. 

As to the Stratonovich integral, since X and V are semimartingales, we can write |P901 Theorem 23, 
page 68] that 

{^j X5T^ = (X • r)^ + i [X, tY = (X • r^) + i [x, r^] = j x5t\ 

Finally, observe that for any process, {X'^Y = X'^ . On the other hand, taking into account that 
l[o,r]^ = l[o,r]^^ and [T,X] = [X,T], we have 

/ ^-^F^ = i[o,r]^ • r + ^ r]^ = i[o,.jX- • F + Q [X, vY^ 

= (X-.Fr+Q[X^F]j ^I^X^ -T+^IXYT]^ =(^jx^5T^ . ■ 
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Proposition 5.6 Let X : M+ x ^ R &e a real valued process. Let {tViI^j^jj be a sequence of stopping 
times such that a.s. tq = 0, t„ < t„_|_i, for all n gN, and sup„gpf t„ = oo. Then, 



X = lim X^" 

ucp 
n—!-oo 



In particular, iJT : x M is a continuous M -valued semimartingale and 7/ G {^I) then, 
J (77, dr) = lim (^J (77, dr)^ = lim J2 J ^ir„,r„^i] {v,dT). 

Proof. Let e > and t G M+. Then for any s G [0, t] one has 

-X\^>e}<Z {t„ < s} C {t„ < <} . 

Hence for any t G K+ 

P({|X-"-X|, >e})<P({T„<t}). 
The result follows because P ({t„ < i}) ^ as n ^ oo since t„ — s- cx) a.s., and hence in probability. 

Let now F be a M- valued continuous semimartingale and 77 G {M). Notice first that (rj, dT) 



because tq = 0. Consequently, by Proposition 15.51 we can write 



and the result follows. 



Proposition 5.7 Let X andY be two real semimartingales. Suppose that X is continuous and Xq = 0. 
Then, for any t G K+, the Stratonovich integral J (l[Q t]Y^ SX is well defined and equal to (^J YdX^ . 

Proof. If J (l[o_t]F) SX was well defined, it should be equal to J (l[o^4]F) dX + i [ip.tji^, ^] . Since 
/ (l[o^t]F) dX is well defined, the only thing that we need to check is that [l[o_t]y, X] exists. On the 
other hand, recall that f [P90l Theor em 12 page 60 and Theorem 23 page 68]) 

J YSX^ - J {l[o,t]Y)dX + ^[Y,Xf ^ J {l[o,t]Y)dX + ^[Y\X]. 

Hence, what we are actually going to proceed by showing that [ip^tjF, X] is equal to [F*, X]. 

Let (T„ = {0 = Tq < Ti < ... < TJ}^ < 00} be a sequence of random partitions tending to the identity 
(in the sense of |P90[ page 64]). Given two real processes X and Y, their quadratic variation, if it exists, 
can be defined as the limit in ucp when n ^ cxd of the following sums 



Let now 



fc„-i 



i=0 
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It is clear that the sequence {Hn}nen converges uniformly on compacts in probability to [F*, X]. We are 
going to prove that there exists such a convergence for the sequence of processes {Gnj^gj^ by showing 
that the elements {Gn)g coincide with (i?„)^, for any s £ R+, up to a set whose probability tends to 
zero as n — > oo. We will consider two cases: 

1. The case s < t. Given a specific i G {0, fc„ — 1}, and recalling that by construction T" < Tl[^^ 
a.s., it is clear that ^(F*)"^^^ — (F*)^' ^ — It'^jAs — Yt^as is different from only for those lo £ D, in 
{T" < s} in which case it takes the value 

5^T4,A. (5.9) 

On the other hand, ^(ijo^tjF)^'"^^ — (l[o,t]F*)"^' ^ is again different from only in the set {T" < s} 
and there it is equal to (|5.9[) . Therefore, (G„)g = {Hn)s whenever s < t. 

2. The case s > t. In this case, - (F*)^"') = ^tAT^^ - Yt^rp which is different from 
only in the set {T" < <}, where it takes the value 

Yt^Tr_^^-YTp (5.10) 

However, in this case (^(l[o,t]F)^'+' - (ifo^tjF*)^' ) = l|j,„^^^^|rtAT/;i -lj7^^<t}FtAT/', which is equal 
to (jgl^ in the set {Tp_^^ < t} (which contains {T;" < t} since T;" < T-'^J, but differs from ((5TII)l in 

Aiit) ■■= {Tr<t<T:x,} 

where it takes the value —It". For any other G i7 not in these sets, ^(ip.fjF)"^'"^^ — (l[o,f]F*)^' ^ (w) = 

0. Therefore, whenever s > t, (Gn)s and (Hn)^ are different only for the ui G ^4" (t). Observe that, 
since t is fixed, only one of the sets {A" (i)}jg{Q is non-empty and, on it, 

(i/„),-(G„), = y,(Xi-XT.). 

To sum up, the analysis that we just carried out shows that for any u G M+ 
sup l(il„). - (G„)J = lA^t) \Yt\ I {Xt ~ Xt'^)\ 

0<s<ti 

for some i G {0, fc„ — I}. If X is continuous, this expression tells us that supo<s<u \{Hn)^ — {Gn)J — > 
a.s. as n — > oo which, in turn, implies that supq<^<„ K-ffn)^ — (G„)^| converges to in probability as 
well. That is, for any e > 0, 



PN sup \iHn),-iGn)J>e}] ^0, asn^oo, 
V Lo<s<u J / 

which is the same as saying that — G„ converges to in ucp. Thus, since G„ = H„ — {Hn — Gn) and 
the limit in ucp as n —>■ oo exist for the both sequences {-ffnlng^ and {Hn — G„}^gpj, so does the limit 
of {G„}„gpj which, by definition, is the quadratic variation [l[o_4]F, X]. Moreover, as (iJ„ — G„) 
in ucp as n —^ oo, 

[Y\X] = lim Hn = lim G„ = [l[o.t]Y,X] , 

ucp ucp L L . J 

n^oo 71 — 'OO 

which conchides the proof. I 
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6 Appendices 

6.1 Preliminaries on semimartingales and integration 

In the following paragraphs we state a few standard definitions and results on manifold valued semi- 
martingales and integration. Semimartingales are the natural setup for stochastic differential equations 
and, in particular, for the equations that we handle in this paper. For proofs and additional details the 
reader is encouraged to check, for instance, with [CW90| lDu96l lE89 l llWSOl ILeG97| [P90] . and references 
therein. 

Semimartingales. The first element in our setup for stochastic processes is a probability space 
(ri, P) together with a filtration {Tt \ t > 0} oi such that contains all the negligible events 
(complete filtration) and the map 1 1 — > Tt is right-continuous, that is, J-t — ne>o -^t+t- 

A real-valued martingale T : R+ — [0, oo) x f2 ^ R is a stochastic process such that for every pair 
t, s £ M+ such that s <t, we have: 

(i) r is J-t-adapted, that is, Tt is -measurable. 

(ii) T,=E[Tt 

(iii) Tt is integrable: i?[|rf|] < +oo. 

For any p e [1, oo), F is called a -martingale whenever F is a martingale and Ft G LP(17), for 
each t. If suptgjj^ E[|Ft|P] < oo, we say that F is -bounded. The process F is locally bounded 
if for any time t > 0, sup{|Fs(tj)| \ s < t} < oo, almost surely. Every continuous process is locally 
bounded. Recall that a process is said to be continuous when its paths are continuous. Most processes 
considered in this paper will be of this kind. Given two continuous processes X and Y we will write 
X = Y when they are a modification of each other or when they are indistinguishable since these two 
concepts coincide for continuous processes. 

A random variable r : ^ [0, +oo] is called a stopping time with respect to the filtration {Tt 
i > 0} if for every t > the set {uj \ t{llj) < t} belongs to Given a stopping time r, we define 

J^r = {A e I A n {r < e for any t £ R+} . 

Given an adapted process F, it can be shown that F^ is -measurable. Furthermore, the stopped 
process F"^ is defined as 

r[ := FfAr := rtl{f<^} + TT-l{t>T}- 

A continuous local martingale is a continuous adapted process F such that for any n e N, 
F'^"1|t-^>o} is a martingale, where r„ is the stopping time t„ := inf{i > | |Ftl = n}. 

We say that the stochastic process F : R+ x ^ R has finite variation whenever it is adapted 
and has bounded variation on compact subintervals of R-|_. This means that for each fixed lu G fl, 
the path t i — > Ft(aj) has bounded variation on compact subintervals of R+, that is, the supremum 
sup {X^iLi l^til'^) ~ rt._j(aj)|} over all the partitions = io < < • ■ • < = ^ of the interval [0,t] is 
finite. 

A continuous semimartingale is the sum of a continuous local martingale and a process with 
finite variation. It can be proved that a given semimartingale has a unique decomposition of the form 
F = Fq + y + A, with Fq the initial value of F, a finite variation process, and A a local continuous 
semimartingale. Both V and A are null at zero. 

The Ito integral with respect to a continuous semimartingale. Let F : R+ x — *■ R be a 

continuous local martingale. It can be shown that there exists a unique increasing process with finite 
variation [F, F]f such that Fj — [F,F]4 is a local continuous martingale. We will refer to [F,F]i as the 
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quadratic variation of F. Given T = Tq + V + A,r' = T'^ + V + A' two continuous local martingales 
we define their joint quadratic variation or quadratic covariation as 

[r, T']t = ^ ([A + A', A + A']t - [A, A]t - [A', A']*) . 

Let {Xn}nef>s be a sequence of processes. We will say that {-'^nlngN converges uniformly on 
compacts in probability (abbreviated ucp) to a process X if for any e > and any t G R+ , 



P^^supjX„-X|^| >ej ^0, 

as n — > oo. 

Following 'P90', we denote by L the space of processes X : ]R_|_ x $7 — *■ R whose paths are left- 
continuous and have right limits. These are usually called caglad processes, which are initials in French 
for left- continuous with right limits. We say that a process A" £ L is elementary whenever it can be 
expressed as 

p-i 

1=1 

where < ri < • • • < Tp_i < Tp are stopping times, and Xq and Xi are J-q and J-V. -measurable random 
variables, respectively such that \Xq\ < oo and \Xi\ < oo a.s. for all i £ 1}. l^^.n+i] is 

the characteristic function of the set (r^jTi+i] = {{t,uj) G i?+ x | t G (r^ (w) , Tj+i (w)]} and Ij-p} of 
{(t, w) G R+ X I i = 0} . It can be shown (see |P901 Theorem 10, page 57]) that the set of elementary 
processes is dense in L in the ucp topology. 

Let F be a semimartingale such that Fq = and X elementary. We define ltd 's stochastic integral 
of X with respect to F as given by 

r "'^ 

AT-F / XdF :=^X,(F^'+i -F^O- (6-1) 
■' 1=1 

In the sequel we will exchangeably use the symbols X ■ F and / XdT to denote the Ito stochastic 
integral. It is a deep result that, if F is a semimartingale, the Ito stochastic integral is a continuous 
map from L into the space of processes whose paths are right-continuous and have left limits (cadlag), 
usually denoted by IS, equipped also with the ucp topology. Therefore we can extend the Ito integral 
to the whole L. In particular, we can integrate any continuous adapted processes with respect to any 
semimartingale . 

Given any stopping time r we define 



/ XdT -.^ {X ■T)r. 
Jo 



It can be shown that (1[o^t-]X) - T — {X ■ Vy = X ■ F"^. If there exists a stopping times (r such that the 
semimartingale F is defined only on the stochastic intervals [0, ^r)) then we may define the Ito integral 
of X with respect to F on any interval [0, r] such that r < Cr by means of X • F"^. 

The Stratonovich integral and stochastic calculus. Given F and X two semimartingales we define 
the Stratonovich integral of X along F as 



j X5T^ j XdT + hx,T]t. 
Jo Jo ^ 
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Let X^, . . . , XP be p continuous semimartingales and / G C'^{M.p). The celebrated ltd formula states 
that 



l,...,xf) = f{x'„ ...,xl) + j2f^^Ml..., x^)dxi 

{Xl...,X^)d[X\X^]s 



fix, 



2 Jo dx'dxi 



The analogue of this equality for the Stratonovich integral is 

fiX^, ...,Xl) = f{Xl ...^XD + Y. f ^Ml...., XP)5Xl. 
An important particular case of these relations are the integration by parts formulas 



/ XdT = iXT)t - {XT)o ~ f TdX- l-[X,T]t, 

Jo JQ ^ 



X6r = ixr)t - {xr)o - / t6x. 

Jo 

Stochastic differential equations. Let F = (r^,...,rP) be p semimartingales with Fq = and 
/ : X ^ a smooth function. A solution of the ltd stochastic differential equation 

p 

dX'=^/;(X,F)dFJ (6.2) 
i=i 

with initial condition the random vector Xo — {Xq, . . . , Xq) is a stochastic process Xt — {X^, . . . , X^) 
such that XI - X'^ ^ Tfj=i lo /](^,r)dFJ . It can be shown [P90l page 310] that for any a; G M« there 
exists a stopping time ^ : x SI ^ E_|. and a time-continuous solution X{t,uj,x) of (|6.2p with initial 
condition x and defined in the time interval [0,({x,ll!)). Additionally, limsupj^^^^ ^-j ||Art(ti;)|| = oo a.s. 
on {C < oo} and X is smooth on x in the open set {x \ Cix,Lu) > t}. Finally, the solution X is a 
semimartingale . 

6.2 Second order vectors and forms 

In the paragraphs that follow we review the basic tools on second order geometry needed in the definition 
of the stochastic integral of a form along a manifold valued semimartingale. The reader interested in the 
proofs of the statements cited in this section is encouraged to check with [E89I , and references therein. 

Let M be a finite dimensional, second-countable, locally compact Hausdorff (and hence paracompact) 
manifold. Given m G M, a tangent vector at m of order two with no constant term is a differential 
operator L : C°° (M) — > R that satisfies 

L (m) = 3/ (m) L [f'] (m) - 3/^ (m) L [/] (m) . 

The vector space of tangent vectors of order two at m is denoted as t^M . The manifold tM := 
UmGM '''mM is referred to as the second order tangent bundle of M . Notice that the (first order) 
tangent bundle TM of M is contained in tM . A vector field of order two is a smooth section of 
the bundle tM — > M . We denote the set of vector fields order two by X2(M). If y, Z G X{M) then the 
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product ZY € X2(M). Conversely, every second order vector field L G X2(M) can be written as a finite 
sum of fields of the form ZY and W, with Z,Y,W G X(M). 

The forms of order two il2{M) are the smooth sections of the cotangent bundle of order two 
r*M [JmeM^M. For any f,g,he C^(M) and L e X2{M) we define da/ £ n2{M) by d2f{L) := 
L[f], and 6^2/ • ^^23 e f22(M) as 

d2/ • d2g[L] := i (i [/g] - fL [g] - gL [/]) . 

It is easy to show that for any Y,Z,WG X{M), 

d2f-d2g[ZY] = ^{Z[f]Y[g] + Z[g]Y[f]) and ^2/ • ^29 [W^] = 0. 

More generally, let a,„, (3.^ G T*^M and choose /, g G C°° (Af ) two functions such that d/(m) = «,„ and 
dg(TO) = It is easy to check that (d/ ■ dg){m) does not depend on the particular choice of / and 
g above and hence we can write am • Pm to denote (d/ • (lg)(m). 11 a, (3 G ri(M) then we can define 
a • /3 G Q.2{M) as (a • (3) (m) := a(m) • f3{m). This product is commutative and C°° (M)-bilinear. It 
can be shown that every second order form can be locally written as a finite sum of forms of the type 
d/ • dg and d2h. 

The d2 operator can also be defined on forms by using a result (Theorem 7.1 in |E89j ) that claims 
that there exists a unique linear operator d2 : fl{M) ~> n2{M) characterized by 

d2 (d/) = ^2/ and ^2 (/a) = d/ • a + fd2a. 



6.3 Stochastic integrals of forms along a semimartingale 

Let M be a manifold. A continuous A/- valued stochastic process X : x ^ Af is called a continuous 
M -valued semimartingale if for each smooth function / G C°°{M), the real valued process f o X is 
a (real- valued) continuous semimartingale. We say that X is locally bounded if the sets {Xsiio) \ < 
s < t} are relatively compact in M for each t G R+, a.s. 

Let X be a Af- valued semimartingale and 9 : x 17 — > t*M be a caglad locally bounded process 
over X, that is, tt o 9 — X , where tt : t*M — > M is the canonical projection. It can be shown (see |E89[ 
Theorem 6.24]) that there exists a unique linear map 6 1 — > J {9,dX) that associates to each such 9 a 
continuous real valued semimartingale and that is fully characterized by the following properties: for 
any / G C°° (Af) and any locally bounded caglad real-valued process K, 

{d2foX,dX)^f{X)-f{Xo), and j {K9,dX) ^ j Kdl^j {9,dX)^ . (6.3) 

The stochastic process J {9,dX) will be called the ltd integral of 9 along X. If a G il2(Af), we will 
write in the sequel the Ito integral of a along X, that is, J{ao X, dX) as / (a, dX). 

The integral of a (0, 2)-tensor & on Af along X is the image of the unique linear mapping h \ — > 
/ b{dX, dX) onto the space of real continuous processes with finite variation that for all /, G C°°{M) 
satisfies 



{fb){dX,dX)^ {foX)d[ b{dX,dX)] and {df ^ dg) {dX,dX) ^ [f o X, g o X]. (6.4) 



If a G f2 (Af ) and X is a semimartingale on Af, the real semimartingale / {d2a,dX) is called the 
Stratonovich integral of a along X and is denoted by / (a, 5X). This definition can be generalized by 
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taking (3 &T*M valued semimartingale over X and by defining the Stratonovich integral as the unique 
real valued semimartingale that satisfies the properties 

(d/, 5X) = f{X)-f (Xo) , and / (Z/3, 5X) = [ Z {X) S ( [ {(3, SX)) , (6.5) 



for any / G C°° (M) and any continuous real valued semimartingale Z. Finally, it can be shown that 
(see [E89l Proposition 6.31]) for any /,g G C°°(M), 

I {df-dg,dX) = ^lf{X),giX)]. (6.6) 
6.4 Stochastic differential equations on manifolds 

The reader interested in the details of the material presented in this section is encouraged to check with 
the chapter 7 in [E89] . 

Let M and N be two manifolds. A Stratonovich operator from M to TV is a family {e{x, y)}x£ M,y£N 
such that e{x,y) : TxM —> TyN is a linear mapping that depends smoothly on its two entries. Let 
e*{x, y) : T*N T*M be the adjoint of e{x, y). 

Let X be a M-valued semimartingale. We say that a A^-valued semimartingale is a solution of the 
the Stratonovich stochastic differential equation 

5Y = e{X,Y)5X (6.7) 

if for any a G f^(iV), the following equality between Stratonovich integrals holds: 

{a,5Y) = j {e*{X,Y)a,5X). 

It can be shown |E89[ Theorem 7.21] that given a semimartingale X in M, a measurable random 
variable Yq, and a Stratonovich operator e from M to A'^, there are a stopping time C > and a solution 
Y of (16. 7p with initial condition Yq defined on the set {(i,ti^) G x | t G [0,C(a;))} that has the 
following maximality and uniqueness property: if ^' is another stopping time such that C' < C ^.nd Y' 
is another solution defined on {(i, lo) G K+ x | i G [0, C'(ci;))}, then Y' and Y coincide in this set. If C 
is finite then Y explodes at time that is, the path Yt with t G [0, Q) is not contained in any compact 
subset of N . 

The stochastic differential equations from the Ito integration point of view require the notion of 
Schwartz operator whose construction we briefly review. The reader interested in the details of 
this construction is encourag ed to check with [E89| . Note first that we can associate to any element 
L G X2(M) a symmetric tensor L G X(M) ® X(M). Second, given x € M and y € N, a linear 
mapping from t^M into TyN is called a Schwartz morphism whenever /{T^M) C TyN and f{L) = 
(/|t^m (8) /|t^a/) (L), for any L G TxM. Third, let M and N be two manifolds; a Schwartz operator 
from M to TV is a family {/(x, y)}xeM,yeN such that f(x, y) : TxM —f TyN is a Schwartz operator that 
depends smoothly on its two entries. Let f*{x,y) : t*N t*M be the adjoint of f{x,y). Finally, let 
A be a M-valued semimartingale. We say that a iV-valued semimartingale is a solution of the the ltd 
stochastic differential equation 

dY ^ f{X,Y)dX (6.8) 
if for any a G ft2(N), the following equality between Ito integrals holds: 

\a,dY) = [{r{X,Y)a,dX). 



Lazaro and Ortega: Stochastic Hamiltonian dynamical systems 



44 



There exists an existence and uniqueness result for the solutions of these stochastic differential equations 
analogous to the one for Stratonovich differential equations. 

Given a Stratonovich operator e from M to A^, there exists a unique Schwartz operator / : tM x 
N — > tN defined as follows. Let 7(t) — {x{t),y{t)) G M x be a smooth curve that verifies 
e{x{t),y{t)){x(t)) = y{t), for aU t. We define f{x{t),y{t))(^LA^(^t)) ■= {Ly{t)): where the second or- 
der differential operators (-/^^(t)) G Tx(t)M and {Ly(^t)) £ '''y(t)N are defined as {Lx(t)) VA ■= ^h{x{t)) 
and (iy(t)) [g] := ^g{y{t)), for any h G C°°{M) and g G C°° (N). This relation completely deter- 
mines / since the vectors of the form span T^(^f-^M. Moreover, the Ito and Stratonovich equations 
6Y = e{X, Y)SX and dY = f{X, Y)dX are equivalent, that is, they have the same solutions. 
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